Requirements Discussion: what is quicli?

[vitiral]

This is a discussion on what the use case and requirements of these two libraries are.

stdcli's goal is basically to make creating a CLI in rust more like creating one in python from a user experience and ergonomics point of view. This means:

• batteries included
• almost everything you need is already imported
  • python, because of it's duct typing and general unsafetly, does not require you to import "something" to use methods on its types. Rust, on the other hand requires you to import a boatload of traits.
  • related: rexport std_prelude in prelude #18
• "flush out" macros that should be in the stdlib like hashmap!
• (not yet in stdcli but planned) provide feature flags to control which libraries you want so you can develop with "full batteries" and then pare everything down as your application dependencies become more clear. For instance, maybe ansi_term, pretty_tables, termstyle and tabwriter are all in the "full batteries" package, but then you can do features = ["style_termstyle"] to only have termstyle

Also related: require that all versions in quicli are of the form >=VERSION so that the user can mostly control which version is used.

What is quicli?

Qui CLI seems to have similar but not completely overlapping goals. I would prefer we merge forces rather than create competing libraries.

I think it should be possible through liberal use of feature flags to accomplish any usecase a user might want. I.e. features = "full_batteries" would depend-on-the-world, but features = "light" would give very minimal dependencies, and everything in between.

[vitiral]

Also: I've got some decent docs in stdcli on why I am exporting stuff -- you might find them useful.

[killercup]

Wow, this turned out way longer than I anticipated. Sorry about that – but thank you very much for making me think about all of this! :)

I would prefer we merge forces rather than create competing libraries.

That would be my preference as well! I do want to keep this crate small, though. And I fear we might want to go in different directions because of that. Or rather, you'll want to go much further than me. So, I hope we can share the first part of the way at least :)

As written on Reddit, my goal is to provide an opinionated set of convenience functions to quickly build CLI apps – And I want to stop there!

This crate already re-exports a bunch of stuff, and I agree that these re-exports make the experience of writing CLIs quickly much more smooth. There are some traits that are very useful, and deserve to re-exported! I want to be very deliberate in what I re-export, though. Let me give some reasoning for this.

First, a disclaimer: I'm not sure what crates/functions/types/traits "typical Rust CLI apps" actually use (I'd love to hear what people are actually using!).

E.g., having Read and Write in the prelude seems like a good idea, but AtomicBool? I've never used that one the last 4 years I've written Rust code. The attitude I wrote this crate with is even more radical, though: I'm pretty sure even Read and Write are only used in a few specific use cases. So, we should try to identify these use cases, and provide convenience functions for those! (I originally wrote that in #18 but moved it here for context.)

Just like I didn't want to re-export env_logger and instead changed the main! macro to initialize the logger automatically, or added simple fs functions, I'd rather introduce some (simple and leaky) abstractions that promise to only be useful 80% of the time instead trying to give the user everything they need. Another example (which I haven't implemented) that works this way: A fn regex_matches(&str, &str) -> CaptureMatches that automatically uses lazy_static and only covers the simplest case (maybe even use a type simpler than CaptureMatches). Have you seen the regex docs? They are wonderful. Sadly, for newcomers/forgetful people/drunk programmers/etc. they are also wonderfully complex.

So, instead of offering the user "everything", I want to introduce some abstractions that are simple to use, and have simple but useful examples. In addition to that, the documentation should point to where to look when you want to use some of these parts more directly/seriously.

stdcli's goal is basically to make creating a CLI in rust more like creating one in python from a user experience and ergonomics point of view.

This is an honorable goal! Rust is not Python, though, and I'm note sure how much I would try to stretch the language to make it feel like it is. I want people to know they are writing Rust, and if that means they get compiler errors telling the to use std::sync::Arc;, that's fine. I was actually thinking about adding some Rust beginner tips to the Readme, like "try .clone() if you get a lifetime error!"

provide feature flags to control which libraries you want so you can develop with "full batteries"

Oh god no. I'm not ready for this library to get so complex after two days in existence! 😄

If you need a "full truck batteries" feature you are not writing "small CLI apps quickly" anymore, and you should surely stop relying on quicli to be your One True Dependency and add some more yourself. I'd rather have an "eject" option to switch from quicli to "all crates imported manually" than add all these crates—including an extern crate kitchensink;.

QuiCLI seems to have similar but not completely overlapping goals.

Yes. This is my conclusion as well.

I'd love to collaborate with you on this. But I don't think we should merge stdcli into quicli. Maybe the other way around? It seems like I am way more conservative in what I want this crate to contains. That may be a good foundation for stdcli to use as a starting point from which you include more crates and features.

[vitiral]

I really like your response! I think it is on the right track.

Here's what I'm thinking... the ecosystem is probably not quite ready for a crate like stdcli. Instead, we should have different crates that clean up the ergonomics of different aspects of developing CLI crates and stdcli could unify them (the non-CLI ecosystem might be able to do this as well, but let's focus on CLI).

Take this crate (quicli) for example, I am (currently spitballing) its core requirements as make the main() experience for writing a CLI in rust simple and clean. This includes an appreciable amount of things:

• Define your command line arguments (with struct_opt)
• Create the Args struct (only one line but still boilerplaty)
• Setup logging: this one has traditionally really bitten rust on the simplicity side. I love the approach you've taken!
• Define your main function with reasonable (and pretty) error handling.

Fulfilling these requirements in an ergonomic fashion is absolutely something that we need! It's not every major requirement of a CLI, so we should probably make other "combining forces" crates. Some examples:

• cli_system: deal with operating system details. Examples include time (chrono), signal handling ctrlc, system libraries (libc), shell variables (shellexpand) and randomness (rand).
• files+directories. Some crates could be path_abs, walkdir, tar
• deserialization and config files: toml, serde_json, serde_yaml, etc
• terminal input/output styling: tabwriter, pretty_tables, termstyle, etc
• There also need to be "generally needed stuff" -- lazy_static, maplit, failure.

I think this crate doing the "generally needed stuff" would be really good, but otherwise it shouldn't touch the other domains. Instead we should create similar crates that combine the above, document them, have integration tests, etc.

And then... stdcli could inherit from all of these and combine them into one crate, with feature flags to disable pieces that aren't needed.

At the end of the day we have simple components that add up to a complete and batteries included experience. Even better, library authors can continue to solve "one little problem" and then have their crate added to one of the conglomerating crates. This is better for both parties -- keeping crates small is really good for the ecosystem, but also being able to have an "off the shelf solution" for general problems is necessary for the ecosystem's continued growth.

[vitiral]

I renamed the title to better reflect what this issue is (I think)

[vitiral]

One of the suggestions from this design: close #14 #11 #6 #2 #1

Maybe keep #5 #15 #4, although things like that belong much more in a crate like stdx... although that is not maintained :(

I think the idea of creating "pieces of an ecosystem" is the right one. If we could design the ecoysystem from a high level we can (hopefully) start to see where the missing peices are and the role of each (conglomerating) crate individually.

[killercup]

That sounds awesome! Very well put.

its core requirements as make the main() experience for writing a CLI in rust simple and clean

That sounds about right!

It'd say it does go a small step beyond that, and I don't really want to stop doing that: I also want to add/export the most common functions you need in a CLI (whatever those may be).

So, in my mind, quicli gives a user the first basic layer of functionally: Add quicli and writing simple CLIs (quickly). It has a read_file helper, but to do more fancy stuff, you need to use std::fs or an external crate. It has logging, but if you want to log structured output to journald, you have do it yourself.

This is where the second layer of crates comes in: It gives you a good selection of fancy tools for specific areas. (I also think that there are some crates out there that are already perfectly usable as "second layer" crates, e.g. serde_json, reqwest, or duct.)

I think this crate doing the "generally needed stuff" would be really good

Wait, do you mean quicli with "this crate"? 'Cause I'm not sure I agree that maplit is generally needed (I've never needed it even though I think it's cool) and I don't see quicli re-exporting lazy-static – except indirectly in a regex function.

we should create similar crates that combine the above, document them, have integration tests, etc.

This is one of the hardest jobs in any ecosystem that promotes small libraries – picking and choosing and integrating a bunch of small building blocks to construct something that is useful and not chaotic.

Do you want to start writing all these crates right now or do you want to gather some feedback of what people ask for and where they find annoyances first? I'd probably go with the latter one but I'm not sure how much experience you've already gathered! :) And I have to admit, I think this is gonna be lot of work.

the ecosystem is probably not quite ready for a crate like stdcli

You right, it is probably too early for a "stable" stdcli – most of the crates you listed are not 1.0 yet and still evolving their APIs. It's hard to say which crates we'll be using next year.

In which way do you want to organize this effort? Also, I'm sure having some kind of "brand" would go a long way to connect the crates together.

[killercup]

Ah, you wrote #19 (comment) while I was typing that :)

I think I agree with most your you choices of what issues to close, but I think I might swap maplit for itertools. I'll let the dust settle for a bit and then see where to go from here.

[vitiral]

I also want to add/export the most common functions you need in a CLI (whatever those may be).

This is a deep dark hole from which there is no escape 😃.

Seriously, once you start going beyond a basic requirement it become impossible (in my experience) to figure out what belongs and what doesn't.

Do you want to start writing all these crates right now or do you want to gather some feedback of what people ask for and where they find annoyances first?

I think some of them could be written immediately -- especially the config one (serde is pretty standard now) and probably the sys one as well. I think this crate is a good example of something that has been a long time coming but which is ready for prime time. No one is going to use anything other than structopt going forward, so tieing it with your logging+main is pretty brilliant and clean.

I think the filesystem needs some work (like the error type you mentioned) but in my opinion it is really close.

The least stable is definitely the "terminal output" one. I don't think there is any stability there, and I've found most solutions to be unergonomic, especially when testing (and haven't yet started using the one I wrote, termstyle, yet -- which is attempting to solve some of the issues).

On maplit specifically

I've found maplit to be a lifesaver when writing tests or the rare time I want to create serialized output by hand. Mostly, as a python user, it's just a huge annoyance to not have a dict at your fingertips when you need it.

[kbknapp]

Just FYI, I'm finally trying to finish up the work to merge structopt into clap proper (assuming my paid work doesn't keep interrupting 😉). This will fix several issues and allow better interop between them. I'll ping y'all once it's ready. Just wanted to mention that since it could somewhat affect these crates going forward.

[moosingin3space]

What about including some sort of configuration management, such as configure?

[vitiral]

@moosingin3space looks awesome!

[vitiral]

I just started the ergo crate and project: making rust's ecosystem more ergonomic, therefore more fun.

https://github.com/vitiral/ergo

I will be creating sub-crates as defined above in the coming weeks. If anyone wants to help out feel free to dig in your heels!

[killercup]

Let's discuss configure in #21.

@vitiral I'm excited for ergo! Ping me if you want me to join :) I can't promise I'll have lots of time to help out, hence I'm probably not of much help on the implementation side, but I know a lot of synonyms for 'ergo' and am thus able to suggest weird module names.

FYI, after seeing how much text I wrote in this issue here, I stared rewriting it as a blog post. You can find a draft of it here if you're interested :)

[vitiral]

@killercup invitation sent, first sub-crate built 🎉

[killercup]

I think we've come to a good conclusion here :)