Inquiry about Data License for Research on Open Source Software Ecosystem

[frank-zsy]

My name is Frank Zhao, a Ph.D. student in Computer Science at Tongji University, China. Our research lab has a strong interest in data analysis of the open-source software ecosystem, and we recently discovered your platform, which contains a wealth of highly valuable data for research. We are very keen to utilize this data for our academic work.

I noticed on OpenCollective that you offer a Data License purchase option. We would like to understand more about this license.

Given that your API has a limit of 5,000 calls per hour, we are curious about the ways in which we can access the data if we purchase the Data License. We have considered the following possible approaches:

Unlimited API Calls: While this would be helpful, it might still require significant development and maintenance effort on our part.
Static Data Share: Providing the data as static files for us to import and analyze would be ideal. However, the challenge here would be how to handle ongoing updates, such as the frequency of updates and how incremental data would be delivered.
Read-Only Database Access: Granting us read-only access to your database for periodic synchronize or simple analysis tasks would be friendly to incremental updates, but it might impose some hardware load on your infrastructure.
These are the methods we have thought of so far. If there are other solutions that might work better, we are open to suggestions.

As our lab’s primary research focus is on open-source software ecosystem data analysis, we are very excited about the possibility of working with your excellent data. We look forward to your response.

[BenJam]

Frank, thanks for getting in touch. We offer a data licence which replaces the CC-BY-SA licence with a CC-Zero licence on the translations provided in the services. This does not, however, include an increase in the API request limit.

We are currently in negotiations with a non-profit to fund work that will allow us to offer traffic prioritisation and increased rate limits but we are currently operating beyond our capacity (serving around 1.2bn requests a month) and cannot offer an increase right now.

On the data imports: we periodically publish data for our most popular services, information on which you can find at https://blog.ecosyste.ms/2023/08/24/open-data-releases.html. Our proposal includes solving some of the challenges you highlight, including synchronising an instance of ecosyste.ms run privately, with the canonical instance we run.

Hopefully that helps, if you'd like to follow up please mail us at hello@(our domain) as we predominantly use GitHub issues for development.

Thanks again.

[frank-zsy]

Hi, thanks for the reply, the open data release is very helpful to our research.

I am wondering if you can update the data to latest version since the recent packages release is one year ago. (https://packages.ecosyste.ms/open-data)

If you can release the open data every month, it could be sufficient for research work, and I think our lab could support your work with a monthly donation as well.

We sincerely hope this work can continue, and that you can also receive the recognition and rewards you deserve.

[frank-zsy]

@BenJam Do you think you can accept USDT or USDC donation to support monthly data release for downstream researches and applications?

[BenJam]

Frank, please get in touch by email. It's possible, if we're able to attribute the funds to a legal entity, for us to accept donations in USDC or T.