Wazua
»
SME
»
Ideas
»
Data analysis of Wazua
Rank: Elder Joined: 2/23/2009 Posts: 1,626
|
Not using any specific one except excel.The rest I just create when the need arises. Uncertainty is certain.Let go
|
|
Rank: Elder Joined: 2/23/2009 Posts: 1,626
|
|
|
Rank: New-farer Joined: 7/24/2012 Posts: 37
|
Loved the piece. I am also in the field of data mining to look cool like everybody else, we call ourselves "Enterprise Intelligence" or simply "EI" Kuna kuchapwo, kuchapana na kujifichio
|
|
Rank: Elder Joined: 7/10/2008 Posts: 9,131 Location: Kanjo
|
Great job ChessMaster. Hope you got a cheque for this analysis would help lady admin to make wiser decisions going forward. i.am.back!!!!
|
|
Rank: Elder Joined: 3/19/2013 Posts: 2,552
|
I am yet to meet a man who can motivate me with money enough to sacrifice my interests.
|
|
Rank: Elder Joined: 3/19/2013 Posts: 2,552
|
Definition of spaghetti code.For educational purposes only Graphs
Code: using System; using System.IO; class ss { static void Main() { string[] bb = {"SCOM","ACCS","UNGA","MSC","ORCH","EVRD","EABL","CARB","BAT","BOC","TCL","OCH","CITY","ICDC","PAFR","CFCI","KNRE","JUB","CIC","BRIT","UMME","TOTL","KPLC","KENO","KEGN","PORT","CABL","BERG","BAMB","ARM","UCHM","TPSE","SGL","SCAN","NMG","LKL","KQ","XPRS","SCBK","NIC","NBK","KCB","HFCK","EQTY","DTK","COOP","CFC","BBK","FIRE","MASH","CMC","WTK","SASN","REA","LIMT","KAPC","KUKZ","EGAD","C%2526G","%5ENASI","%5EN20I"}; string inn = "http://live.mystocks.co.ke/graphs/"; string im = ".Chart?mspgsid=3e186ffb|4322|1364211724.4653;r="; string ie = ";f=w;t=avg;v=ovr;corp=0;id=1364212103737"; FileStream ff = new FileStream("nn.txt", FileMode.Create, FileAccess.Write); StreamWriter ww = new StreamWriter(ff); int[] pp = { 30, 60, 90, 180, 270, 360, 420, 540, 720, 1080, 1800, 3600, 7200 }; string dd; for(int i = 0; i<bb.Length; i++) { for(int j = 0; j< pp.Length; j++) { dd = inn+bb[i]+im+pp[j]+ie; ww.WriteLine(dd); } } ww.Close(); ff.Close(); } }
List
Code: using System; using System.Runtime.InteropServices; using System.Net; using Excel = Microsoft.Office.Interop.Excel; using HtmlAgilityPack; using System.IO;
namespace MyStocks { class Program { static int rr = 2; static int cc = 1; static void Main(string[] args) { for (int pp = 1; pp <= 31; pp++) { string dd; string ff = "201301"; if (pp < 10) { dd = ff + "0" + pp; } else { dd = ff + pp; } string str = "http://live.mystocks.co.ke/price_list/" + dd; WebClient aa = new WebClient(); var webget = new HtmlWeb(); var doc = webget.Load(str); var nod = doc.DocumentNode.SelectNodes("//tr[@class='row r1']"); var noda = doc.DocumentNode.SelectNodes("//tr[@class='row r0']"); Excel.Application app = new Excel.Application(); Excel.Workbook Wbook = app.Workbooks.Add(1); Excel.Worksheet sheet = (Excel.Worksheet)Wbook.Worksheets.Add(); sheet.Cells[1, 1] = "Name"; sheet.Cells[1, 2] = "Low-12"; sheet.Cells[1, 3] = "High-12"; sheet.Cells[1, 4] = "Low-Day"; sheet.Cells[1, 5] = "High-Day"; sheet.Cells[1, 6] = "Price"; sheet.Cells[1, 7] = "Previous"; sheet.Cells[1, 8] = "Change-Price"; sheet.Cells[1, 9] = "Change-Perc"; sheet.Cells[1, 10] = "Volume"; sheet.Cells[1, 11] = "Adjusted Price";
for (int yy = 0; yy < noda.Count; yy++) {
var gh = noda[yy].SelectNodes("td"); int gg = gh.Count; for (int i = 1; i < gh.Count; i++) { if (i == 8) { i = 11; sheet.Cells[rr, 8] = (sheet.Cells[rr, 6].Value) - (sheet.Cells[rr, 7].Value); sheet.Cells[rr, 9] = (sheet.Cells[rr, 8].Value) / (sheet.Cells[rr, 7].Value) + "%"; cc = 10; } sheet.Cells[rr, cc] = gh[i].InnerHtml; cc++; } rr++; cc = 1; } for (int yy = 0; yy < nod.Count; yy++) {
var gh = nod[yy].SelectNodes("td"); int gg = gh.Count; for (int i = 1; i < gh.Count; i++) { if (i == 8) { i = 11; sheet.Cells[rr, 8] = (sheet.Cells[rr, 6].Value) - (sheet.Cells[rr, 7].Value); sheet.Cells[rr, 9] = (sheet.Cells[rr, 8].Value) / (sheet.Cells[rr, 7].Value)*100+"%"; cc = 10; } sheet.Cells[rr, cc] = gh[i].InnerHtml; cc++; } rr++; cc = 1; } rr = 2; cc = 1; Wbook.SaveAs(@"C:\Finale\" + dd + ".xls"); app.Quit(); Marshal.ReleaseComObject(sheet); Marshal.ReleaseComObject(Wbook); Marshal.ReleaseComObject(app); } } } }
|
|
Rank: Elder Joined: 3/19/2013 Posts: 2,552
|
How do you make a crawler that understands concepts?With NLP you can identify words and such.A concept is about a philosophy of something.This is about perception and attitudes.How does one code a system that can read information and tell you the gist of the story?Like a personal assistant.
I'm thinking if I can make one which can read businessdaily and other news feeds integrate it with my signals system I can automate my trading system by embedding my analysis skills in it.The potential is ridiculous.Now let me meditate on that.
|
|
Rank: New-farer Joined: 1/10/2011 Posts: 29 Location: nyahururu
|
ChessMaster, thank you for starting this topic . This is why I love wazua it challenges me!! To my problem: I am trying to parse the faNtaSEy > 2011 Experts Leaderboard table at faNtaSEy > 2011 LeaderBoard. Since the table is paginated the crawler can't get past the 1st page. ideas on how to go about it. Also welcome, are interesting ideas on what to do when I get the data. The opposite of love is not hate but apathy. So too, the opposite of courage is not fear but mediocrity
|
|
Rank: Elder Joined: 3/19/2013 Posts: 2,552
|
g-mi wrote:ChessMaster, thank you for starting this topic . This is why I love wazua it challenges me!! To my problem: I am trying to parse the faNtaSEy > 2011 Experts Leaderboard table at faNtaSEy > 2011 LeaderBoard. Since the table is paginated the crawler can't get past the 1st page. ideas on how to go about it. Also welcome, are interesting ideas on what to do when I get the data. Thank you.Let me look at it and I'll get back to you.The data I focus on is the trades.I focus on who is buying,what they are buying and how fast they sell.This is so I can see who has good timing and consistent in trading.I also check the quantity bought and the quantity traded during the day to see if the trade would have been possible in the market.
|
|
Rank: Elder Joined: 3/19/2013 Posts: 2,552
|
g-mi wrote:ChessMaster, thank you for starting this topic . This is why I love wazua it challenges me!! To my problem: I am trying to parse the faNtaSEy > 2011 Experts Leaderboard table at faNtaSEy > 2011 LeaderBoard. Since the table is paginated the crawler can't get past the 1st page. ideas on how to go about it. Also welcome, are interesting ideas on what to do when I get the data. I suggest you download a software called fiddler.It can help decipher web requests with the help of Mozilla
Code: POST http://www.wazua.co.ke/investor/fantasygame.aspx?fsg=2 HTTP/1.1 Host: www.wazua.co.ke User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:14.0) Gecko/20100101 Firefox/14.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip, deflate DNT: 1 Connection: keep-alive Referer: http://www.wazua.co.ke/investor/fantasygame.aspx?fsg=2 Cookie: __utma=1.1794717367.1364293920.1364293920.1364293920.1; __utmb=1.1.10.1364293920; __utmc=1; __utmz=1.1364293920.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none) Content-Type: application/x-www-form-urlencoded Content-Length: 56980
__EVENTTARGET=ctl00%24ContentPlaceHolder1%24grvFantasy&__EVENTARGUMENT=Page%242
I've tried posting the whole request but it doesn't fit the number of allowed characters!Its that big.Get fiddler so you can see for yourself.The first few lines you will see something like Quote:__EVENTTARGET=ctl00%24ContentPlaceHolder1%24grvFantasy&__EVENTARGUMENT=Page%244 This is for page 3 Quote:__EVENTTARGET=ctl00%24ContentPlaceHolder1%24grvFantasy&__EVENTARGUMENT=Page%243 The difference is the last 2,3,4 which are the pages of the Experts challenge.What you do is let the crawler scan the first page so it can get the number of total pages.From there when requesting the rest of the pages configure the crawler to use a POST request which will use an iteration to change that specific digit.I haven't done it but have used the same method for other things.Try it and tell me how it works.
|
|
Rank: Elder Joined: 3/19/2013 Posts: 2,552
|
Wow,this is going to be interesting.Computers don't understand words.Just found a new obsession.
|
|
Rank: Chief Joined: 8/4/2010 Posts: 8,977
|
g-mi wrote:ChessMaster, thank you for starting this topic . This is why I love wazua it challenges me!! To my problem: I am trying to parse the faNtaSEy > 2011 Experts Leaderboard table at faNtaSEy > 2011 LeaderBoard. Since the table is paginated the crawler can't get past the 1st page. ideas on how to go about it. Also welcome, are interesting ideas on what to do when I get the data. Are you aware that this fantasy market does not capture new listings e.g. Britam, cfci, cic, longhorn etc... $15/barrel oil... The commodities lehman moment arrives as well as Sovereign debt volcano!
|
|
Rank: Elder Joined: 3/19/2013 Posts: 2,552
|
Humans live with uncertainty and are accustomed to living with and acting in relativity.Computers work on absolutes.Humans make relations to objects and ideas by utilizing their senses and imagination.Computers have no reference. Objects in language exist in context and reference to something else.Thus creating boundaries or groupings which computers can identify and a relationship matrix of some kind can be built upon that. Thinking on the line of children.Why do they continue to ask why?Curiosity can only be built upon the boundaries of an established knowledge base.You can't ask what you don't know. There are many ways to ask a question.Why didn't they win or why did they lose?How does a computer know which one to ask and what to base it on.There is also a problem with socializing.How would it learn to be social,build rapport,deal with sarcasm(Reminded of Sheldon here ),jokes and quotes.Also how would they deal with adjectives because they are very subjective. Given a framework,an AI system can gain knowledge independently,comprehend it,solve problems but the true test lies in synthesizing knowledge independently.Lateral thinking would be interesting,like a philosophical AI.
|
|
Rank: Elder Joined: 2/26/2012 Posts: 15,980
|
@symbols any experience with cellphone data "There are only two emotions in the market, hope & fear. The problem is you hope when you should fear & fear when you should hope: - Jesse Livermore .
|
|
Rank: User Joined: 9/6/2013 Posts: 1,446 Location: In a house
|
nostoppingthis wrote:@chessmaster, which software you using? natafuta mwalimu wa SAS Talk nicely. Otherwise, you'll pay through the nose. I'm ready to teach you.
|
|
Rank: Elder Joined: 3/19/2013 Posts: 2,552
|
|
|
Wazua
»
SME
»
Ideas
»
Data analysis of Wazua
Forum Jump
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.
|