A Journey at Data.gov by nickumia

Disclaimer: The thoughts and ideas conveyed in this blog are those of the author and do not reflect those of GSA, REI, Data.gov or any other organization that the author is part of.

Executive Summary

It's been quite a journey over the last 2 years. From my onboarding ticket til now, a lot has changed about the world, Data.gov, life in general and about me as well. Let me start by saying that this is not a "farewell" or "final words" post because I will continue to be active (in varying capacities) in Data.gov. In the days to follow, it will mostly be guidance on issues and background research on design choices that the Data.gov team will face. However, long-term, I would love to contribute in an open-source fashion once some additional work is done to solidify the direction. Data.gov is a complex program. Even if you consider the metadata problem that Data.gov meets the needs of to be easy, the non-technical aspects of connecting people with data cause sooo many intricacies of development. I'm proud to say that I have supported Data.gov and I know Data.gov is super important in fostering collaboration on a national level. In recent times, the challenges of supporting Data.gov revolve around fortifying the underlying infrastructure and processes to ensure adequate time and energy can be spent towards the real work.

We've tried to organize the wealth of knowledge and experience in the Data.gov Wiki, the Data.gov google drive and across sooo many repos; however, the nature of the problem is that there's just so much to know and be familiar with. I'll make lots of references throughout this and I hope that most of it will be preserved into the future; however, please reach out to @nickumia if anything is broken and I can help find things too!

History

A good place to start is the list of issues I've contributed through over the years. I spent 'some' (read: a long) time revisiting all of my issues and labeling them to give context to find knowledge bits that are embedded in the foundation of the Data.gov codebase. There is a natural hierarchy of systems of systems.

flowchart TB
    subgraph Data Applications
    direction TB
    CKAN{{CKAN}}
    Catalog>catalog.data.gov]
    Inventory>inventory.data.gov]
    Harvest[/Harvesting\]
    Egress[/Egress Proxy\]

    CKAN --> Catalog
    CKAN --> Inventory
    Catalog --> Harvest
    Catalog --> Egress
    Inventory --> Egress
    end

    subgraph Social Branding
    direction TB
    Datagov>data.gov]
    11ty[/11ty Static Generator\]
    Datagov --> 11ty
    end

Loading

Table of Systems: https://github.com/GSA/data.gov/blob/main/SYSTEMS.md Cloud.gov APP Tracker: https://docs.google.com/spreadsheets/d/1RlN2ihGIsZDU4Ho4KzHHookNpAb8TQ0fz20Xz9Yiq1A/edit#gid=0