Output column ordering deterministic?

[gravesee]

When I pass a list of features into final_vars is the ordering deterministic? I would like to apply hamilton in a pyspark mapInPandas function and need to specify the return schema of the pandas dataframe. It is unclear to me whether I can depend on the ordering of the final_vars list. Great framework!

[skrawcz]

🤔 It definitely should be:

1. We create a dictionary and insert into it based on the order of final_vars. https://github.com/stitchfix/hamilton/blob/main/hamilton/driver.py#L175-L177
2. If you're using the default driver (and thus result builder), then a pandas dataframe will be created here https://github.com/stitchfix/hamilton/blob/main/hamilton/base.py#L174
3. Which delegates to pandas, and according to their docs If data is a dict, column order follows insertion-order. should follow the insertion order (source).

Otherwise to plant the thought, we'd love contributions to our examples folder, or a blog on how you're using Hamilton with pyspark.