OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Scrape web site generated by Javascript

  • Thread starter Thread starter bagshawa
  • Start date Start date
B

bagshawa

Guest
I think this is a real challenging one!

I write a website for my local football league, www.rdyfl.co.uk , and include javascript code snippets from the F.A's Full-Time system where we generate our fixtures, linking in tables fixtures recent results etc.

For another feature I want to add to the site I need to scrape the 'Upcoming Fixtures' for each agegroup and division but when I examine the source I have two problems.


  1. The fixtures content is generated by javascript and therefore I need to see the generated source and not just the source.


  2. When I view the generated source using Firefox the team names are actually further javascript links and not the name itself.

I basically want to somehow download the fixtures on a regular basis and write then to a mysql database ?

I have asked the F.A. and they have no more options available to access the data ?

Having never coded for scraping before can anyone point me to a simple solution or does anyone fancy the challange?

<p>I think this is a real challenging one!</p>

<p>I write a website for my local football league, www.rdyfl.co.uk , and include javascript code snippets from the F.A's Full-Time system where we generate our fixtures, linking in tables fixtures recent results etc.</p>

<p>For another feature I want to add to the site I need to scrape the 'Upcoming Fixtures' for each agegroup and division but when I examine the source I have two problems.</p>

<ol>
<li><p>The fixtures content is generated by javascript and therefore I need to see the generated source and not just the source.</p></li>
<li><p>When I view the generated source using Firefox the team names are actually further javascript links and not the name itself.</p></li>
</ol>

<p>I basically want to somehow download the fixtures on a regular basis and write then to a mysql database ?</p>

<p>I have asked the F.A. and they have no more options available to access the data ?</p>

<p>Having never coded for scraping before can anyone point me to a simple solution or does anyone fancy the challange?</p>
 

Latest posts

Top