-
Notifications
You must be signed in to change notification settings - Fork 16
Get Meetup.com info #10
Comments
Dropping by to leave some Hacktoberfest cheer (it ain't a PR but it's something and the issue at hand caught my attention.) Scraping a site like Meetup. com should be fairly easy at it's core. The question is on the complexity of the task is implementation. Namely the location of the meetups and info needed. By default the site appears to navigate to the meetups it thinks are in you local area, so if it's for their that super easy. But, if the info needs to be gotten for a different location (which I presume), probably the easiest way would be url or javascript manipulation. The info should be pretty easy to scrape though, no super crazy page elements to be seen; basic xpaths like this should work. Meetup Title: //section[@id="techEventsShelf"]//span[contains(@Class ,"sectionTitle")] Of course if you need meetup descriptions that might be more complex (and add longrer runtime). I apologize in advance if this is all repeat on stuff you already know, don't mean to redesign the wheel :) |
Does this need something like a data warehouse where this scrapes once a night and then we have our own DB we can query? Do we run this nightly and rebuild the site (if its a static site) every night if there's a change? We sure do not want to scrape every time someone visits the site; we want minimal strain against the meetup platform, as to not cause any unnecessary attention & for speed. |
I have a project going for this |
I do not feel we will bring any "strain" to their servers with this but storing the data / pulling once a day would be more than enough. |
They definitely have the bandwidth for a daily pull and you can store it in something simple like Firebase Real-Time Database |
Meetup.com closed their API this year unless you pay for a pro account. We need a way to scrape the data from all the tech meetups in town and display it on our calendar.
Possibly also have a preview function (like slack or text preview) for the event webpage
The text was updated successfully, but these errors were encountered: