Webscaping with some ML - I need some help please?

[BinstedMike]

I am trying to rebuild all the share buy-back daily data for all LSE FTSE 100 and 250 listed companies by using the LSE's Regulatory News Service. The problem I am having is that each page that the data is on varies so trying to build a sucessful tool is proving hard. Can anyone help me please?

A) I can get a complete list of companies who have done share buy-backs by running this filter, and then downloading all the results and pivoting by company name:
https://www.londonstockexchange.com/news?tab=news-explorer&headlinestypes=1,2&indices=UKX,MCX&period=custom&beforedate=20230325&afterdate=20210328&freetext=transaction%20in%20own%20shares

B) I would like to build a tool that takes each name and builds this filter (example BP plc) by simplying adding the company name to the filter above, but doing it for each company from the list in A
https://www.londonstockexchange.com/news?tab=news-explorer&headlinestypes=1,2&indices=UKX,MCX&period=custom&beforedate=20230325&afterdate=20210328&freetext=transaction%20in%20own%20shares&namecode=bp%20plc

C) Then scaping data from each URL in the company specific filter in B- to collect from eg: https://www.londonstockexchange.com/news-article/BP./transaction-in-own-shares/15890857
the company name, trade date, total number of share bought, total shares average price, broker executing the transactions

As you can see in this case for BP there is no aggregate total number of shares, so we need to either collect the share bought on each venue in the top table, so the sum of LSE, BXE an CXE shares bought Number of Shares purchased: | 5,170,596 | 940,108 | 3,290,379. Howeve on some other company pages this aggregated data is already summed.
Then we need to calculate the weighted average price for all the shares bought by multiplying the shares bought on each venue, by the price that venue, across all the venues, and then dividing by the total number of shares bought. Again on some pages this maths is already done for you.

Can anyone please point in the right direction as to how I can build a tool that can solve for this. Ideally I am looking to build a database for the universe of companies, with each days total number of share bought, average price and broker. Going back as long as LSE RNS history will allow. Then running this script periodically to up date once a week or month so that the database builds on this history (rather than relying on LSE RNS feed, as that is only 3 years scrolling, so you loose a day of history every day)

Any thoughts as to if this has already been solved, or if someone can help me would be greatly appreciated!

[dgtlmoon]

I dont think anyone is going to solve this for you for free, it sounds like a commercial project

[BinstedMike]

Thank you @dgtlmoon - do you have any suggestions as to who I shold talk to regarding such a project?

[dgtlmoon]

@BinstedMike it's possible, you can hit me up on my commercial support email [email protected] , thanks!