Delete all the data from amundsen

[kartik1718]

Hey, I have ingested and published the data on amundsen. And I am able to see all the tables and tags etc on the amundsen UI. I want to delete all the data or some specific table data. Any sample or example would be appreciated. Thanks in advance.

[DeepakBhatt1312]

Hi, You can delete the data from graph db backend. And reindex elastic search.

[VoLKyyyOG]

Hey there, hopefully this can answer it in future if anyone sees it.

Assuming you've used docker to run Amundsen:

1. In terminal (SSH into the service - I use VS Code since it can forward the ports automatically), run curl -v http://localhost:7474/browser/ and then curl -v http://localhost:7687.
2. Now, run curl -v http://localhost:7474/browser/ again and open it into your browser (if you are not using VS Code you might have to do sth else here)
3. In the Neo4J interface, login with credentials. By default it is neo4j and test.
4. Run the following CypherText query: MATCH (n) DETACH DELETE n

If you get a memory error, there are a couple alternatives:

1. Find a way into the docker container and delete the Neo4j graph.db
2. Use the Neo4jStalenessRemovalTask as a job in your loader.py and just set it to remove all dates that are not today. Here's an example:

def remove_stale_data():
    """
    A simple job that removes all stale data.
    The date specified in the config is used to determine if data is stale.
    """
    # remove stale data job
    task = Neo4jStalenessRemovalTask()
    job_config_dict = {
        'job.identifier': 'remove_stale_data_job',
        'task.remove_stale_data.neo4j_endpoint': neo4j_endpoint,
        'task.remove_stale_data.neo4j_user': neo4j_user,
        'task.remove_stale_data.neo4j_password': neo4j_password,
        'task.remove_stale_data.staleness_max_pct': 80,
        'task.remove_stale_data.target_nodes': ['Table', 'Column'],
        'task.remove_stale_data.job_publish_tag': DATE # consider all dates that is not today as stale
    }
    job_config = ConfigFactory.from_dict(job_config_dict)
    job = DefaultJob(conf=job_config, task=task)

    return job