Scraping Data from Covers, Creating an NHL Database

[KennyPowers]

I'm looking for a way to quickly create a database from the NHL information on Covers from the last 5-10 years. For now I guess I am just interested in the scores of each period as well as the moneyline and U/O.

I do have quite a bit of programming experience, mostly in Java and VB but I haven't done any web-based programming such as scraping or data mining before.

Any Java libraries you've used to scrape data that might be useful or links to websites with easier information to parse would be greatly appreciated.

[TomG]

Don't go back further than the lockout and you might want to skip a year or two after the lockout when teams were still adjusting to changes in strategy. There are enough NHL games in a season where sample size shouldn't be an issue.

[AngryPlanets]

I used WebHarvest for my Java scraping. Easy and effective.

[MonkeyF0cker]

If you know VB.NET, you can use either a Webbrowser control, Webclient, or the HttpWebRequest class.

Personally, I wouldn't scrape from Covers. I'd use a combination of several line services and the NHL website.

[strixee]

I do have quite a bit of programming experience, mostly in Java and VB but I haven't done any web-based programming such as scraping or data mining before.

Data mining has nothing to do with web-based programming
For scraping I recommend using PHP+MySQL, it's especially useful if you want to use some output accessible online (plus it can run on a shared hosting).

As MonkeyF0cker said, Covers isn't a good source to scrape from. I'm actually shocked, that I see odds just for 1 side like here covers.com/sports/odds/linehistory.aspx?eventId=47344&sport=nhl ! Each sportsbook has different vig, so you don't know what the other side price was.
Covers is too US sports oriented service.

[mathdotcom]

Data mining has nothing to do with web-based programming
For scraping I recommend using PHP+MySQL, it's especially useful if you want to use some output accessible online (plus it can run on a shared hosting).

As MonkeyF0cker said, Covers isn't a good source to scrape from. I'm actually shocked, that I see odds just for 1 side like here covers.com/sports/odds/linehistory.aspx?eventId=47344&sport=nhl ! Each sportsbook has different vig, so you don't know what the other side price was.
Covers is too US sports oriented service.

If you look historically where they're getting their numbers you can infer the other side with reasonable accuracy

[strixee]

Here you can get regular season ML odds from Pinnacle since 2009 http://strixee.mysbrforum.com/blog/1...-pinnacle.html

[mbs4]

If you look historically where they're getting their numbers you can infer the other side with reasonable accuracy

Where does covers get their numbers from?

[KennyPowers]

Thanks everyone for the replies. I will look into WebHarvest and those VB.NET classes. And thanks for the GREAT link strixee, that, along with the scores of each period should definitely start me in the right direction.

[Jrod124]

you will quit within a week

[KennyPowers]

How come? You don't think NHL is worth the time? Or you don't think I'm worth the time

[durito]

How come? You don't think NHL is worth the time? Or you don't think I'm worth the time

Probably doesn't even understand what you are trying to do.

[a4u2fear]

ive got all odds, results, win streaks going into game, goals for/against going into game for 2011 season.

[a4u2fear]

ive got all odds, results, win streaks going into game, goals for/against going into game for 2011 season.

this includes home/away odds, over/under odds.

[357vegas]

Has anyone scraped data from Jeff Sagarin usatoday pitching? I'm looking for a little help to gether data from this site. I have no clue how to, I just know how to use the data.

[newbottles]

this includes home/away odds, over/under odds.

I am new to this, so pardon me if this is an ignorant question. Is this data from some sort of manual or semi-automated daily collection during the season? Or from an an automated data collection approach?

[a4u2fear]

I am new to this, so pardon me if this is an ignorant question. Is this data from some sort of manual or semi-automated daily collection during the season? Or from an an automated data collection approach?

manually performed it and manipulated. i did however find out that you can do a much easier web query through Excel

[degengamble]

Easiest way to do it is with excel - web queries / vba

[allin1]

outwit hub is very good for beginners