How do I extract data from multiple related web pages in Android using Jsoup? -

-

well, have been working in app display news headings , contents site http://www.myagdikali.com

i able extract data 'myagdikali.com/category/news/national-news/' there 10 posts in page , there links other pages 1,2,3... myagdikali.com/category/news/national-news/page/2.

all need know is, how extract news every possible pages under /national_news ? possible using jsoup ?

till code extract data single page is:

public view oncreateview(layoutinflater inflater, viewgroup container,                          bundle savedinstancestate) {     view rootview = inflater.inflate(r.layout.fragment_all, container, false);     int = getarguments().getint(news);     string topics = getresources().getstringarray(r.array.topics)[i];      switch (i) {         case 0:             url = "http://myagdikali.com/category/news/national-news";             new newsextractor().execute();              break;             .....   [edit] private class newsextractor extends asynctask<void, void, void> {    string title;  @override protected void doinbackground(void... params) {      while (status == ok) {         currenturl = url + string.valueof(page);           try {             response = jsoup.connect(currenturl).execute();             status = response.statuscode();             if (status == ok) {                  document doc = response.parse();                 elements urllists = doc.select("a[rel=bookmark]");                 (org.jsoup.nodes.element urllist : urllists) {                      string src = urllist.text();                      mylinks.add(src);                  }                 title = doc.title();             }         } catch (ioexception e) {             e.printstacktrace();         }     page++;      }     return null;     }

edit: while trying extract data single page without loop, can extract data. after using while loop, error stating no adapter attached.

actually loading extracted data in recyclerview , onpostexecute this:

    @override     protected void onpostexecute(void avoid) {         layoutmanager = new linearlayoutmanager(getactivity());         recyclerview.setlayoutmanager(layoutmanager);          myrecyclerviewadapter = new     myrecyclerviewadapter(getactivity(),mylinks);         recyclerview.setadapter(myrecyclerviewadapter);       }

since know url of pages need - http://myagdikali.com/category/news/national-news/page/x (where x page number between 2 , 446), can loop through urls. you'll need use jsoup's response, make sure page exists (the number 446 can changed - believe increases).
code should this:

final string url = "http://myagdikali.com/category/news/national-news/page/"; final int ok = 200; string currenturl; int page = 2; int status = ok; connection.response response = null; document doc = null;  while (status == ok) {     currenturl = url + string.valueof(page);  //add page number url     response = jsoup.connect(currenturl)             .useragent("mozilla/5.0")             .execute();  //you may add here useragent/timeout etc.     status = response.statuscode();     if (status == ok) {         doc = response.parse();         //extract info. need     }     page++; }

this of course not working code - you'll have add try-catch sentences, compiler you. hope helps you.

edit:
1. i've editted code - i've had send useragent string in order response server.
2. code runs on machine, prints lots of ????, because don't have proper fonts installed.
3. error you're getting android part - views. haven't posted piece of code...
4. try add useragent, might solve it.
5. please add error , code you're running original question editting it, it's more readable.


Comments

Post a Comment

Popular posts from this blog

qt - Using float or double for own QML classes -

-
i've created components , helper methods using c++ , qt used in qml gui. of methods calculate gui elements (size, transformations, scene graph items creation). target arm 32 bit using opengl / eglfs render output. opengl using float , qml using real (defined double ). what should use in c++ do not lose precision? if use qreal ( double ) shouldn't lose precision. if git grep float in qtbase.git , can find out bit more: dist/changes-5.2.0-**************************************************************************** dist/changes-5.2.0-* important behavior changes * dist/changes-5.2.0-**************************************************************************** dist/changes-5.2.0- dist/changes-5.2.0- - qt compiled qreal typedef'ed double on dist/changes-5.2.0: platforms. qreal float on arm chipsets before. guarantees more dist/changes-5.2.0- consistent behavior between platforms qt supports, binary dist/chan...
Read more

Create Outlook appointment via C# .Net -

-
i need create outlook appointment using microsoft.office.interop.outlook , while can work locally on own workstation , works if run web application via server's browser, not work when connect server externally. hence, think might permission issue. i changed application pool identity " localsystem " don't access denied error. unfortunately, doesn't work. the app behaves if appointment created, appointment doesn't appear in outlook. here include file on top of aspx.cs page using outlook = microsoft.office.interop.outlook; here code using pop appointment. outlook.application apptapp = new outlook.application(); outlook.appointmentitem appt = apptapp.createitem(outlook.olitemtype.olappointmentitem) outlook.appointmentitem; appt.subject = txtfirstname.text + " " + txtlastname.text; appt.body = txtcomment.text; appt.alldayevent = false; appt.start = datetime.parse(txtreminderdate.text + " 8:00 am"); appt.end = datetime.p...
Read more

ios - Swift Array Resetting Itself -

-
i have 2 arrays: pointhistory , datehistory . have nstimer , , when timer runs out, action called. in action appending int pointhistory , , current date string datehistory . here timer action: func timeraction() { let steppervalue = int(stepper.value) employee.numberofpoints = steppervalue networking.saveemployee(employee, completion: { (error) -> void in if error != nil { let alert = uialertcontroller(title: "error", message: "there error updating \(self.employee.name)'s points", preferredstyle: .alert) let alertaction = uialertaction(title: "ok", style: .cancel, handler: nil) alert.addaction(alertaction) self.presentviewcontroller(alert, animated: true, completion: nil) } }) var dateformatter = nsdateformatter() dateformatter.dateformat = "mm-dd-yyyy" dateformatter.timezone = nstimezone.localtimezone() var date = nsdate() var ...
Read more