to_yaml() does not honor ordered_box

[philipsd6]

In [1]: from box import Box
In [2]: ordered_box = Box(ordered_box=True)
In [3]: ordered_box.zlast = 'first'
In [4]: ordered_box.afirst = 'last'
In [5]: ordered_box.keys()
Out[5]: ['zlast', 'afirst']  # correct, as expected
In [6]: ordered_box.to_yaml()
Out[6]: 'afirst: last\nzlast: first\n'  # not ordered as expected!

YAML really looks better when ordered logically, not arbitrarily!

[cdgriffith]

Ouh, good idea. I think it will require passing the object through the python builtin OrderedDict somehow, so will have to look into this. Thanks!

[philipsd6]

FWIW, PyYaml 5.1+ can handle this with the sort_keys parameter. Prior to this version, it sorted keys alphabetically unconditionally, so there was no easy way to get insert ordered output. Here's a demo of the new behavior:

In [1]: from box import Box                                                                                                                                   
In [2]: b = Box(last="1", first="2", an_arbitrarily_long_key_that_should_absolutely_be_last="3", ordered_box=True)                                            
In [3]: b.keys()                                                                                                                                              
Out[3]: ['last', 'first', 'an_arbitrarily_long_key_that_should_absolutely_be_last']
In [4]: print(b.to_yaml())                 # this gets sorted alphabetically >_<
an_arbitrarily_long_key_that_should_absolutely_be_last: '3'
first: '2'
last: '1'
In [8]: print(b.to_yaml(sort_keys=False))  # this is ordered correctly!!!
last: '1'
first: '2'
an_arbitrarily_long_key_that_should_absolutely_be_last: '3'

So unless you want to automatically add sort_keys to your to_yaml() method (after automatically detecting if that parameter is available in your PyYaml dependency) no change is really necessary, as I can manage the order myself as an end user.

[cdgriffith]

That's great to know! I will have to see if such a simple thing also exists with ruamel.yaml and if so easy enough to include on ordered boxes all the time.

Thanks for figuring that out and glad you can get it to work for what you need already!

[cdgriffith]

Sadly it looks like doing this in ruamel is much more cumbersome https://stackoverflow.com/questions/40226610/ruamel-yaml-equivalent-of-sort-keys and I don't think is worth setting as the default method for everyone's use case.

[philipsd6]

Sorry to update a long closed issue, but... now that ruamel.yaml is the default, I needed to figure this out again. The easiest way as an end-user to get creation-ordered YAML output is now:

In [1]: from box import Box                                                                                                                                   
In [2]: b = Box(last="1", first="2", an_arbitrarily_long_key_that_should_absolutely_be_last="3")                                            
In [3]: b.keys()                                                                                                                                              
Out[3]: ['last', 'first', 'an_arbitrarily_long_key_that_should_absolutely_be_last']
In [4]: print(b.to_yaml())                 # this gets sorted alphabetically >_<
an_arbitrarily_long_key_that_should_absolutely_be_last: '3'
first: '2'
last: '1'
In [5]: from ruamel.yaml import RoundTripDumper
In [6]: print(b.to_yaml(Dumper=RoundTripDumper))  # this is ordered correctly!!!
last: '1'
first: '2'
an_arbitrarily_long_key_that_should_absolutely_be_last: '3'

Sadly, using the RoundTripLoader/RoundTripDumper doesn't retain comments, which is one of the major reasons for using ruamel.yaml!

[jimkohl]

First of all love the concept of normalizing different data intake formats for conversion. Thanks for your efforts!

Have been experimenting with lots of different packages and once its time to get to the yml output step
kept running into this, sort_keys for pyyaml fixed the default sorting behavior there which is so counter intuitive,
but I guess its historical from a python key sorting standpoint.

So when I got to using this package I was stuck w sorted data but thank you to @philipsd6 for this tip.
This tip should be hoisted up into the to_yaml usage section.

One of the things that makes python so awesome is to be able to quickly drop in packages and move right along.
Default options are very important to reduce friction. Unexpected behavior creates them.

IMO the basic usecase / expectation is to preserve original order and should be the default.
I would consider sorting to be a specialized usecase. And consistency across conversions would be helpful.
For example, to_json gets it right - it preserves the order.

As is this is a rough edge in the "easy drop in" department and I think really should be addressed as a higher priority.
Thank you for an otherwise great package and I'm happy to help in testing.