Wazua this one is for you.Incomplete and shoddy but it wasn't meant for display. I'm sure you can figure it out.Here is the link for
HAP the html parser I used.All that is needed is Javascript understanding especially DOM and XPath.The rest is C#.
Code:
using System;
using Excel = Microsoft.Office.Interop.Excel;
using HtmlAgilityPack;
using System.Net;
class parse
{
static int rr = 2;
static int cc = 1;
static public void Main()
{
string str, topic, starter, replies, views, lPost, tLink;
int pp = 9; //topic pages 26 -talk2us,24-life you get the drift
Excel.Application app = new Excel.Application();
Excel.Workbook Wbook = app.Workbooks.Add(1);
Excel.Worksheet sheet = (Excel.Worksheet)Wbook.Worksheets.Add();
sheet.Cells[1, 1] = "Topic";
sheet.Cells[1, 2] = "Topic Starter";
sheet.Cells[1, 3] = "Replies";
sheet.Cells[1, 4] = "Views";
sheet.Cells[1, 5] = "Last Post";
sheet.Cells[1, 6] = "Thread Link";
str = "http://m.wazua.co.ke/forum.aspx?g=topics&f="+pp+"&p="+"1";
WebClient aa = new WebClient();
var webget = new HtmlWeb();
var doc = webget.Load(str);
var noda = doc.DocumentNode.SelectSingleNode("//span[@class='pagecount']");
var Pages = doc.DocumentNode.SelectSingleNode("//span[@class='pagecount']");
string pages = noda.InnerText.Replace(" Pages", "");
int Pagel = Convert.ToInt32(pages);
Console.WriteLine("Total Pages " + Pagel);
for (int i = 1; i <= Pagel; i++)
{
str = "http://m.wazua.co.ke/forum.aspx?g=topics&f="+pp+"&p="+i;
doc = webget.Load(str);
var Topic = doc.DocumentNode.SelectNodes("//a[@class='post_link']");
var Starter = doc.DocumentNode.SelectNodes("//a/@id");
var Replies = doc.DocumentNode.SelectNodes("//td[@class='topicReplies']");
var Views = doc.DocumentNode.SelectNodes("//td[@class='topicViews']");
var LPost = doc.DocumentNode.SelectNodes("//td[@class='topicLastPost smallfont']");
var TLink = doc.DocumentNode.SelectNodes("//a[@class='post_link']");
int count = Topic.Count;
Console.WriteLine("page " + i);
for (int j = 0; j < count; j++)
{
topic = Topic[j].InnerText;
sheet.Cells[rr, cc] = topic;
cc++;
//Topic starter
cc++;
replies = Replies[j].InnerText;
sheet.Cells[rr, cc] = replies;
cc++;
views = Views[j].InnerText;
sheet.Cells[rr, cc] = views;
cc++;
lPost = LPost[j].InnerText.Replace(" ", "");
if (lPost.Contains("AM"))
{
lPost = lPost.Replace("AMby", "AM by");
}
else if (lPost.Contains("PM"))
{
lPost = lPost.Replace("PMby", "PM by");
}
sheet.Cells[rr, cc] = lPost;
cc++;
tLink = TLink[j].Attributes["href"].Value;
tLink = "http://wazua.co.ke"+tLink.Replace("amp;","");
sheet.Cells[rr, cc] = tLink;
rr++;
cc = 1;
}
}
}
}
Let me share how I would have made the app without your approval.I would have used a java html parser for android phones and HAP for windows phones.No option for J2ME. I would ride on the mobile platform of Wazua since its lighter and only used the normal version when necessary.
Logins and posts would have used POST method.Layout can be easily replicated on the phone or changed to what one desires.To automate checking for new posts I would use a timer on the allposts link by comparing the last refresh with the latest one.This would check who posted but more specifically the time.An option for timer refresh rate can work.Same concept can also be used to track specific threads or section so someone is informed on what they want.
When I would encounter images and such,I would rely on the phone's browser.It would be a simple independent solution that would leave the website as is.The parsers would simply fetch and process the information which the app processes and organizes.Also thought of a IM feature where members can chat on the side with each other or groups but I think that would have meant making the app use client/server technology.
Another feature I wanted to attempt was a news reader which is possible for some sites e.g.BD,Standard,Nation,East African as well as some international ones actually a lot.The trick is the paragraphs.When someone posts a link you can still utilize the same parser to fetch the paragraphs from the website thus extract and display the information in the same app.Over time one can increase the methods for parsing different websites.
Hope this helps.Wish you luck on your endeavors.