Add basic documents regarding user research #28
Conversation
|
(Nice work correcting all my typos though, generally). |
|
@sadielbartholomew - thanks, this looks good. We should definitely discuss this, and do it, and try to be open-minded about any feedback received from users. However I think we should probably be prepared to find that the vast majority of users are strongly influenced by what they've used already (as you noted) and aren't really aware of other options, or have rather trivial suggestions compared to the big issues that we have to grapple with (how to display huge graphs that show part of an ongoing cycling workflow effectively etc.). However_2, I suppose I should be open minded myself and see what comes out of this! |
docs/user-research-prompts.md
Outdated
| #### My proposals: | ||
|
|
||
| * Use a contained location (repo or directory) in GitHub to record all UR as | ||
| it is accumulated. |
There was a problem hiding this comment.
But once it is all accumulated and we are not collecting any data, can we maybe move it to an issue, maybe in cylc-admin?
There was a problem hiding this comment.
We will probably need to sift through it to pick out concrete, self-contained & realisable requests for improvements & new features etc., but it would definitely be a good idea to move such useful feedback as relevant to the Issue tracker, which is our main way of keeping up-to-date on that. 👍
docs/user-research-prompts.md
Outdated
| it is accumulated. | ||
| * Either make the repository private to the core Cylc team *or* do not mention | ||
| that such a UR log exists or indeed its location (it is very unlikely users | ||
| would look for or find & then read it). |
There was a problem hiding this comment.
I think if we decide to change something in cylc-ui, especially if it is something very different from what we had in the previous GTK UI, maybe having an issue in cylc-admin with the data/rationale explaining why we are doing it this way, could be helpful?
There was a problem hiding this comment.
It could definitely be helpful, good suggestion.
docs/user-research-prompts.md
Outdated
|
|
||
| * Beware of asking *leading questions*! e.g. "*Do you find the GUIs slow?*". | ||
| The best way to avoid this is by asking questions that are | ||
| [direct](https://www.usertesting.com/blog/direct-questions/) i.e. open-ended. |
There was a problem hiding this comment.
The links in this proposal are very interesting! Thanks @sadielbartholomew ! Especially useful to me that do not have much experience with UI/UX.
There was a problem hiding this comment.
Me neither, actually, apart from reading some blog posts & attending the two-day course! There is an awful lot of very helpful material online, but it is a matter of finding what is relevant to us, as it is a whole field in itself.
@kinow: one day I will manage to clean up my notes from the UX course. They are actually up already, but only in the "raw" format as copied up electronically straight from my written notes, & may not make much sense. But if you are happy to look at them in the raw state, they are here.
docs/user-research-prompts.md
Outdated
| ("pain points") & work-arounds for them, "flows" (the step-by-step path | ||
| followed through the UI), habits, opinions, needs, ideals. | ||
| * We do not know what collected UR data (& "metadata" i.e. background | ||
| information) might be useful going forward. |
There was a problem hiding this comment.
Sounds like this UX course was really good! Good points.
docs/user-research-prompts.md
Outdated
| * It would be helpful if we could all freely add to & edit documents holding | ||
| UR, with knowledge that we won't undo or block other people's amendments. | ||
| * It is *not wise to disclose the store of UR with any users* we are collecting | ||
| input from, as they will be influenced & biased by other people's comments. |
There was a problem hiding this comment.
Could we disclose after the collection is done?
There was a problem hiding this comment.
Certainly, though I am sceptical as to whether users will be particularly interested in the results!
|
Have very little experience with UX, and only did some UR via telemetry in web apps, without being involved in the next steps after the data was collected. Looks like a good document. I am not sure when it should be done. Should we ask before we build the initial UI views, or should we build the basic views to show that what we have is working, then list pros & cons and what we can and cannot do (e.g. we didn't know what filtering involved with GraphQL and Vue.js until a few days ago), and then start collecting user feedback? Or even better, if we could perhaps do it some sort of annual or continuous process. Maybe every UM conference have a session where each site can contribute their users feedback/data? Anyhow, +1 from me, great stuff @sadielbartholomew ! It will be great to have some direction on which way to go with the UI. |
| @@ -66,7 +66,7 @@ UI SERVER and UI *JAN. FEB. MAR. APR. MAY. JUN. JUL. AUG. SEP. OC | |||
| Manage multiple suites | | | | . | ...|...o| | | | | | | | / | | |||
| UI: Tornado GraphQL WebSocket | | | | | .| .. | .. |...o| | | | | | \ | | |||
| WS: ZeroMQ | | | | | .|..o | | | | | | | | / | | |||
| WS UI Workslow State Views | | | | | | | | | | | | | | \ | | |||
|
Can we post "User Research" questions to the Discourse group, and encourage loads of users to read them and respond? |
+1 Whatever is the outcome, we can summarize it later in an issue/document somewhere and use for reference. |
Great idea, & judging by the numbers of 👍's on the comment, many of us agree. Has this been set up yet (I can't see it but it could be there & I am missing it)? If not, I am happy to get it added. Just to clarify the status of this PR (sorry, I have just returned today from some leave), I am going to add in a small document with some points summarising the outcomes of a meeting the UK team had last week, & then this should be ready to go in. I'll get this done tomorrow UK-time (I left my notes at home so I have to bring them in). In the most recent update, I've created a directory dedicated to user research, including two main files for logging user interactions and insights from these, to keep that all contained & shared. |
|
Haven't seen anything about it in the Discourse group yet @sadielbartholomew |
@hjoliver & @kinow (& @cylc/core), in our (MO) team meeting today, we agreed that we would discuss what questions to pose to users via Discourse when you are in Exeter next week. So I will not add any user feedback threads to Discourse yet. If anyone has some time & the desire, it would be great if they can have a think about what might be suitable. In the team meeting we discussed some ideas, & seemed to agree loosely that it would be good to ask (some directed) questions regarding thoughts (e.g. what is liked & disliked) about:
Ideally those two items can be segregated somehow to try to minimise the chance that our ideas & plans influence (e.g. limit the scope of) the users' own vision & ideas. Some questions that arose from the brief discussion:
|
sadielbartholomew
changed the title
|
Update on this PR: I have a few things to finalise, now that some PRs relating to user research are up in parallel. This is mainly:
|
|
That's great Sadie!
I think design and functionality. Some ideas that may require changes to cylc-flow would lead to new discussion elsewhere, but as we have a major release, I think it would be a good by-product, and coild help drive some of our design for the new ui.
Not sure. But I think ot would be good to show participants our new web gui first, and explain briefly the ui server & hub roles. They are probably familiar with the old GUI. And may already have some opinions for improvement. So I think if they see the direction we are taking, they may offer some feedback that is easier to translate to our current work. |
sadielbartholomew
force-pushed
the
research
branch
3 times, most recently
from
bb09849 to
1e664de
Compare
|
Now updated (with conflicts resolved) & ready for review (though I understand it is a busy few weeks so that this may not get reviewed for a while). |
|
Oops, sorry, I did not realise this had acquired merge conflicts since my previous comment. I will squash in a resolution to these now. |
sadielbartholomew
force-pushed
the
research
branch
from
1e664de to
95121fe
Compare
|
@hjoliver @matthewrmshin can I please bump this to get it reviewed soon (at least before any other |
@sadielbartholomew - sorry can you remind me - this is PR is mainly background/planning info to inform the user questions doc, right? (which you've posted to Riot as PDF) ... in which case I'm happy for it to be merged as-is, and tweaked later if nec. I haven't had time to completely re-read this today. |
|
Thanks @hjoliver. In response:
Yes.
No, it is in fact separate to the user questions document for Discourse. It is about User Research in general going forwards, stemming from an initial meeting the Met Office team had on the topic. There shouldn't be anything controversial in there, but I understand if it can't be reviewed for a while as you are super busy. It just seems other |
|
(yes the bigger PRs tend to get passed over when time is short, unfortunately!). OK I'm going to take advantage of my remaining jet lag to read this now.... |
|
Ah yes, so this is also about the Cylc/Rose boundary moving - which is an important point! |
|
@sadielbartholomew, OK, I read through again very quickly. This is very comprehensive, really good stuff for us to think about it, and it would undoubtedly be great to have this kind of information about the "user experience" of Rose and Cylc across the board. I don't think I disagree with anything you've written. My only hesitation is, how much effort will it be to gather this information, and what will we do with it? I can certainly see it informing better documentation and user training efforts. But for development, I'm a bit concerned that we don't actually have the time or effort or intention to back up such a comprehensive User Research effort with changes to our planned architecture and UI designs. What if users said they think "rose suite-run" should not be migrated to Cylc for instance (hypothetical, hopefully unlikely!) - or that they actually prefer the old local client/server architecture and to hell with the modern web components - what could/would we actually do about that? In reality it would become an education effort by us: we'd have to spend time explaining to users why we think we need to do it, rather than adjusting our plans to fit their stated preferences. So, that's my concern about a) effort required to gather the info; and b) what we could realistically do with it. But that said, it's a great, comprehensive guide to discussion/thinking on all things UX/UR if we had a large enough team to really do it justice ... and on that basis I have no objection to merging it to cylc-admin. Does that make sense? |
|
(And of course - I have to keep reminding myself of this! - it's quite possible that we will learn some things to inform our development that we had not anticipated). |
|
OK, two approvals so I'll merge this ... but still interested in your response to my comment above @sadielbartholomew. |
|
Sure @hjoliver, sorry for the delay responding, though if you don't mind I will answer it later this week as I am busy preparing for & giving user training today & tomorrow. In short, though, I fully understand & we all seem to share those concerns here; my initial proposal was more of an ideal on how we could approach things. |
@cylc/core: we need get started on user research ASAP given that the backbone of the front-end is now set up so that we will start developing (& perhaps connecting as frameworks) the UI components shortly. The UK team will start conducting this at the Met Office, but the best way to do it is up for debate. If those in the Southern Hemisphere have any thoughts it would be great to hear them!
On that note, this PR adds a document summarising key aspects we need to think about before we get going in earnest. My hope is that we can discuss as relevant on Riot.im.
I've formatted the considerations & correspondingly the document into numbered questions, my hope being that we can stick to these questions (else add-in new questions) to help organise our collective thoughts, & perhaps use the numbers as shorthands. Please try to do this!