Custom directives / plugins

[srobbin]

Currently, Vapid has seven content types—text, html, choice, date, image, link and number—otherwise known as "directives." Having a limited number of options was an intentional design decision, so one could master the basics quickly, and build sites without continually referencing documentation. That said, it's easy to imagine there will be a desire for more directives, either as a means of connecting to third-party services or supporting more complex UI/data structures. There have already been a few submissions that suggest so (#4, #6, #49).

I'm opening this issue so we can consolidate the threads, and discuss:

1. What kinds of custom directives we'd like to build (e.g. {{photos type=slideshow}} or {{buy type=paypal currency=usd}}), so we can more fully understand the context.
2. How we'd need to modify the code to support them.

To the second point, the existing directives have a pattern of inheriting from a BaseDirective, so they can define what their dashboard input looks like, how they're rendered, and what options they accept. Hopefully, this is something we can improve and extend for our needs.

For the sake of staying focused, let's hold off on discussing how someone might install custom directives—that's an implementation detail that we can deal with later.

Looking forward to chatting with you about this.

[walidvb]

Hey @srobbin thanks for your consideration!

I appreciate and agree with keeping vapid simple and as bare-bone as possible.

1. I don't think that presentational elements have any place in the core, so type=slideshow sounds out of scope to me. Elements I could imagine should correspond to input types(select, checkbox, etc). I'm not sure what the buy directive would do?
   I think image could benefit from processing, so maybe a processing=200x200 would allow to have consistent image sizes.

Additional tags I can think of:

• type=tag, would allow to query the db against a specific tag.
• type=reference, to reference another record. useful for prev/past section, or related sections

2. From the experience gained in Fields multiple #56 (adding multiple instance of an image field):

Files processing is handled parallel to the normal request body. It was no issue previously, as content[fieldName] would be overwritten anyways. Introducing multiple input required handling things differently(particularly, adding a _destroy checkbox for each record). I believe my implementation would cover the other directives.
(If this last answer is out of scope, let's move it to the PR.)

• directives should have a process: (input) => valueToStore method, allowing to manipulate the response before storing in db.
• BaseDirective should have a inputDestroy method, returning the destroy checkbox when appropriate.

most importantly, thanks for this game changer!

[seekersapp2013]

I think I agree with @walidvb the input names should be the W3C input types. Like type=button, type=checkbox, type=password, type=radio. That way its less confusing for anyone starting out. The idea of type=slideshow is good, because it simplify a lot of things.

[walidvb]

@seekersapp2013 i didn't mean to necessarily restrain to the W3C input types, just to stick to basics, and leave 3rd party integrations and presentational functionality to custom directives, in the form of plugins or smth.

Regarding type=slideshow, are you saying we should have it? What would be the difference with image?

Last, for 2.: perhaps it could be handy for directives to allow custom render=(value)=>value or render=renderFunction?

[seekersapp2013]

I'm saying that it will be easier for an HTML beginner to understand when we use the W3C spec. instead of having a custom name types for the same thing. ie. it will be easier to have type=radio than have a custom name that renders a radio button.

About the type=slideshow, I would be excited to see something like this eg.
{{#section slideshow multiple=true limit=5 required=false}}
{{image type=slideshow src=image1.jpg}}
{{image type=slideshow src=image2.jpg}}
{{image type=slideshow src=image3.jpg}}
{{image type=slideshow src=image4.jpg}}
{{/section}}

This code or something similar should be able to render a slideshow on the page.

[walidvb]

I'll let @srobbin step in, but i'm not 100% i follow.

Is this a template?

{{#section slideshow multiple=true limit=5 required=false}}
	{{image type=slideshow src=image1.jpg}}
	{{image type=slideshow src=image2.jpg}}
	{{image type=slideshow src=image3.jpg}}
	{{image type=slideshow src=image4.jpg}}
{{/section}}

{{#section}} are multiple by default. Why the repetition of {{image type=slideshow}}? What would type=slideshow be?
I'm not sure I get it :/

Still, I could definitely imagine a slideshow directive to exist, only it would be a custom one(which would essentially use a {{slide type=image multiple=true}} directive, and have some javascript to create the slideshow)

[seekersapp2013]

It will even be simpler that way, I'm just kind of sold out to the idea of vapid been "intentionally simple" and I think making the syntax relate-able to HTML syntax will make it much more simpler. I will be looking forward to this feature. And also looking forward to the hosting service. I received a mail today.

[srobbin]

@walidvb: Great feedback. I appreciate you moving this conversation along. Answers and thoughts inline:

I don't think that presentational elements have any place in the core, so type=slideshow sounds out of scope to me.

Agreed, I don't think something like type=slideshow belongs in core either. The hope is that custom directives will let people experiment freely, and give this project an outlet for doing things outside of its proposed scope. I imagine some of them will be good though, and we might consider merging them in down the road.

type=reference, to reference another record. useful for prev/past section, or related sections

YES.

I think image could benefit from processing, so maybe a processing=200x200 would allow to have consistent image sizes.

YES. Maybe that's something we could incorporate with type=image. See #65.

directives should have a process: (input) => valueToStore method, allowing to manipulate the response before storing in db.

Currently, BaseDirective has a serialize method which is meant to serve a similar purpose. I'm curious to know if you think there's a way to extend it to serve more needs, including destroy. We'd probably need a deserialize method too.

I'm not sure what the buy directive would do?

It was just an example, but I could imagine something like it existing, where the directive adds a PayPal button, complete will all the fiddly bits required to generate a PayPal form.

Regarding type=slideshow, are you saying we should have it? What would be the difference with image?

I think the difference is that custom directives might also allow things like:

• The ability to inject CSS/JS into the page (as you mention in a later comment). I say this acknowledging there would need to be a follow-up discussion about security.
• The ability to set global preferences/config. e.g., The type=paypal example might require you to set your PayPal email address under Settings.

@seekersapp2013: I appreciate your input about keeping Vapid easy for beginners, for folks who have HTML knowledge. We'll definitely try to strike that balance.

[walidvb]

@srobbin happy to do so ^^, really appreciate the concept of this. Reminds me of when I was using tumblr to build websites, to avoid dealing with CMS maintenance.

What's your take on tags/taxonomy?

Agree on #65 reg image processing.

Currently, BaseDirective has a serialize method which is meant to serve a similar purpose. I'm curious to know if you think there's a way to extend it to serve more needs, including destroy. We'd probably need a deserialize method too.

I can't find this currently being called anywhere? In any case, easy add, and then it should likely be there that the multiple file is being handled(although it might be tricky to reimplement in custom directives)
Destroy could be as a reserved word in content[fieldName][_destroy][i] unless there's a more elegant solution

I think the difference is that custom directives might also allow things like [injecting assets and global settings]
Agreed, although I don't think this is a required initial step.

• early vapid plugins could require adding the assets to the template(although vapid might not serve them? 🤔)
• (public)Settings could be given in the template file(or a partial) or in an ENV variable.

[srobbin]

@walidvb Sorry for the slow reply—I've been on holiday for the past week or so.

Re: serialize, it's currently being called here, although it doesn't do much right now. I think I originally added it to ensure that the number directive stored values as a number instead of a string.

Agreed, I think making _destroy a reserved word, and standard flow, is a good idea.

[walidvb]

no probs, so have i.

So, how should we go about this? I have some projects that could use this, but I think we should move forward with it.
You want to do some plugin system? Also, reg multiple images, considering the files are handled kind of separately, would you still change my PR?

Last, maybe we could expose both an endpoint to upload images(returning the URL), AND expose the function saveFile, in the case someone wants to use it in a custom directive?

[srobbin]

@walidvb I'd like to leave this conversation open for a bit longer, to get feedback from others. Custom directives are going to be a significant feature, and I want to make sure we do it right, and that it accommodates as many use cases as possible. Regarding timing, I think that this might be something tackled in the New Year.

To your other point, I hope that we'll be able to reuse some of your ideas/code from the PR. I appreciate all of the thought and effort that you've put into this.

[treshenry]

What kinds of custom directives we'd like to build

"I need a calendar where I can put upcoming engagements and events" is a HELLA frequent request. Throwing it in here as another use case for custom directives. Once the directives API is available this is probably the first thing I'll write.

Given the above conversation it sounds like this might be:

{{#section events calendar=true}}
  <em>{{when type=date}}</em>
  <h3>{{title}}</h3>
  <p>{{location}}</p>
{{/section}}

However, maybe a custom directive can declare itself as something like a section?

{{#calendar}}
  <em>{{when type=date}}</em>
  ...
{{/calendar}}

I dunno. But at a minimum a calendar is just a different way of showing a repeating section that contains a date. There may also be a need to turn on/off some interaction modes for users, for things like a summary view or filter/search based on a date range, etc. Doing a quick search for wordpress calendar plugins is... eye opening.

On that note: has there been any thought on how site creators can easily find/consume, and developers can share, custom directives? I'm thinking something like the VSCode marketplace. Might be out of scope for this particular issue but some way for site creators to easily find and ingest plugins is an important part of the UX for extensibility. Number of downloads and voting are critical features that help reduce the chances of 50 different calendar plugins.

vapid install calendar

As always, ❤️

[walidvb]

I think extensibility and plugin architecture is the point indeed of this thread.
As to where these should exist, I don't see why npm or yarn wouldn't do the job, as in the end they are likely to simply be node modules.
As for the exposure, it sounds like Vapid forums is a place dedicated for that, although a (unofficial) page on the wiki of the github is my preferred option.

Reg. calendar, how can this not be done via a simple repeating section?
Also I haven't looked right now, but last i checked most calendar solutions were js-based(at least to display the dates with no events).
To plug the data into a js function, you can output your data as JSON in your template.

Keep in mind that ftm directives declare a form and a display, filtering/searching is a different topic IMHO.

[treshenry]

@srobbin I made the rookie mistake of, "this shouldn't be discussed here BUT..." so before I respond to the discover/share topic above I wanted to check with you on where/how you want to handle that convo.

calendar, how can this not be done via a simple repeating section

Let's take a step back. I think there's a mix of usage and implementation in this thread that's confusing things a bit (at least it is for me). The following may already be defined somewhere else. If so please point me at it.

For the sake of clarity I see three relevant roles to this conversation (any person/people/organization could wear one or more of these roles):

• Extension Developer - Develops, updates, and owns the the extension/plugin/directive (herein extension). ED wants a nice, clean, well documented extension API.
• Site Author - Consumes the extension for use in authoring a site. SiA wants to easily find and add extensions to a site she's working on and she wants the usage of an extension to be simple, idiomatic, and well documented.
• Client - Hires SiA to create a site, generates feature requests, and manages the content of the site through the dashboard. Clie wants the dashboard for an extension to be easy to understand and fit the problem domain.

Let's re-frame a calendar extension through the lens of these roles:

ED is creating a calendar extension. He needs to consider both SiA and Clie in the design and implementation. For example, he doesn't want SiA to have to know the internal workings of the calendar's JSON structure in order to use it. He would also like Clie to have a nice experience when managing the content of the calendar so he needs to think about how it looks in the dashboard as well.

SiA doesn't want to spend a bunch of time figuring out a calendar plugin. She's got limited time because Clie, as an up-and-coming DJ in the Berlin scene, didn't have a lot of money to spend on a website. SiA wants something that comes with everything out-of-the-box and she needs it to pick up the styles already defined for the site without a lot of fuss.

Clie has a giant list of upcoming gigs and just wants to get them into the site quickly. Also, that gig at Berghain this week got moved to next Friday and she needs to let her fans know.

--

Hopefully this helps to clarify boundaries and use cases. So yes, the calendar implementation could, under the covers, be JS that consumes JSON produced by the extension, however, that should be opaque to both SiA and Clie. And if that's the standard extension paradigm it should be easy to implement for ED and feel like a single API surface - not something bolted on. I think considering the dashboard experience for an extension is also useful, even if it's considered out-of-scope for whatever the v1 of extensions looks like.

Yes? No?

[srobbin]

@ohnoimdead, @walidvb Thanks for keeping this thread moving. Agreed, this is a complex topic and needs clarification.

Just to reframe a little bit, it's helpful for me to think of custom directives as having one or more traits: those that simplify the presentation layer, and those that simplify the data entry layer. For example, @walidvb's multiple image uploader improves data entry of images, but may choose to render simple <img> tags on the front-end, allowing the user to decorate as needed; and @ohnoimdead's calendar directive may be able to lean on existing form fields, but might generate a fully functional front-end calendar (a la FullCalendar). A photo slideshow directive may choose to customize both.

I hope that by talking about the needs of Site Authors, we'll be able to identify API requirements for Extension Developers. For example, in thinking about a calendar plugin, we've identified a need for pagination and JSON data feed (for toggling between months).

With that pattern in mind, I see this thread as being focused on Site Author needs and custom directive ideas. Once we've identified technical requirements, we can move into one or more pull requests to talk about implementation.

That said, maybe it would be helpful if we came up with a template for people to submit custom directive ideas. e.g.,

Name
Calendar

Description
Editors enter a title, date and location into the dashboard, and the front-end renders a calendar (a la FullCalendar). Vistors can toggle between months, search events, and subscribe to a iCal feed.

Directive type (presentation, data entry, or both)
Presentation

Sample code

{{section events type=calendar}}
  <em>{{date type=date format="%B %-d, %Y"}}}</em>
  <h3>{{title}}</h3>
  <p>{{location}}</p>
{{/section}}

Requirements

• Inject CSS and JS into front-end
• Pagination
• JSON data feed

What do you think?

---

Re: directive consumption, I think that's something we can discuss down the road, but FWIW I like the idea of a curated repo/marketplace vs. npm. In my opinion, curation is a better way to ensure quality and security.

[treshenry]

Love it! Starting with the Site Author experience is perfect as that is the most likely to inform everything else. I also like the request template which will help provide some structure for thinking about extensions. It might need a bit of explanation for someone coming at it cold though. "Presentation" vs "data entry" might not make sense to someone that hasn't read this thread.

100% in agreement on extension curation. Using npm/GitHub as the backend store for extensions is great, as that makes it easy for Extension Developers to share and update, but without features like voting, usage counts, and search, the Site Author experience will suck.

[walidvb]

100% agree with approaching this with a Site Author perspective.

Templates is also nice.

Reg curation, while I agree it is helpful, I think it would be a shame that developers with custom requirements are not able to add their own directives, without the need of approval of admins.

Last, one would need to inject CSS and JS both into front-end and dashboard, no? Or at least allow the dashboard to hook into the plugin.
Drupal has quite a nice way of handling that, where modules can declare hooks, and the core calls each module's hooks(if they exist), where the hook can process and/or alter data.

Not quite sure what data consumption means in terms of server change, though?
As far as the calendar is concerned, the data can be output as json and consumed front-end by the directive's code.

[stale]

This issue has been automatically marked as stale after 90 days of inactivity. It will be closed if no further activity occurs.

[amiller-gh]

Ref: #157

The same plugin system could be used for discovering custom directives. There is plenty of prior art for this in other frontend build plugin systems – ex: Eyeglass, Ember-CLI, Vue, etc.

A vapid plugin could provide custom directives in a /www/directives folder. They would be automatically discovered, merged into the parent project (giving priority to locally defined directives of the same name Object.assign style), and made available in the parent project's templates.

Side note: I like how these two issue numbers are exactly 100 issues apart.