Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve performance of spatial.import() for large data sets #385

Open
jamesdhope opened this issue Aug 12, 2021 · 0 comments
Open

improve performance of spatial.import() for large data sets #385

jamesdhope opened this issue Aug 12, 2021 · 0 comments

Comments

@jamesdhope
Copy link

I ran an import of the London OSM data from Planet on a 2 Core machine with 4GIG RAM. OSM Nodes and Ways imported within 3-4 hours. After 30 hours Relations were still being imported. In the end I had to kill the process.

It was reassuring that the import ran without memory overflow on such a small machine (possibly using SWAP i'm not sure) but the speed at which relations were being imported was the main issue for us and would likely be too slow to maintain a large OSM dataset. Indeed, one option would be to run this on a larger machine and further testing would be required to see if this would be performant enough for our needs.

It would be good to see if further optimisation of the import routine could be made to improve performance particularly with regard to OSM Relations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant