Traversing collections & paginated collections?

[brabster]

Hi there, nice library!

Does traverson support traversing a collection of items with the same rel? Even better, a paginated collection following "next" links? (I'm thinking of HAL here eg. http://stateless.co/hal_specification.html, "Representing Multiple Links With The Same Relation")

I couldn't see anything in the documentation mentioning collections like this so I thought I'd ask!

[basti1302]

traverson-hal supports collections/multiple links with the same relation.

From the docs:

You can also pass strings like 'ht:post[name:foo]' to the follow method to select links (which share
the same link relation) by a secondary key. Because multiple links with the same link relation type
are represented as an array of link objects in HAL, you can also use an array indexing notation
like 'ht:post[1]' to select an individual elements from an array of link objects. However, this is not
recommended and should only be used as a last resort if the API does not provide a secondary
key to select the correct link, because it relies on the ordering of the links as returned from the
server, which might not be guaranteed to be always the same.

You can also use the array indexing notation 'ht:post[1]' to target individual elements in an array of
embedded resources.

There is no special support for paginated collections with next rel. How would that look like?

[travi]

I'm interested in where this goes as well. I think that my initial thought would be that, if a match was not found under _embedded and a next link was available in _links, the next page would be requested and searched for a match. Rinse, lather, repeat until either a match is found or no next link is provided (there are no more pages).

[basti1302]

That sounds like a pretty reasonable strategy and a nice feature for traverson-hal.

Right now, I'm just not completely sure about the specifics. I'll give an example to explain where I'm a bit confused: If one document had some _embedded documents, say one with key foo and one with key bar and also a link relation next with an URL, how would we know that the next link is related to foo and not to bar?

Can we come up with a concrete example, that is, one or two hal documents, an example Traverson snippet that works with this API and the expected result of the Traveson call?

[brabster]

Hey, sorry I didn't get back to you on this, totally forgot I mentioned it until I saw an email this morning!

I was talking about a resource like the one on page 7 of the hal spec, reproduced below with only the pagination details:

{
     "_links": {
       "self": { "href": "/orders" },
       "next": { "href": "/orders?page=2" }
     },
     "_embedded": {
       "orders": [{
           "_links": {
             "self": { "href": "/orders/123" }
           },
           "total": 30.00
         },{
           "_links": {
             "self": { "href": "/orders/124" }
           },
           "total": 20.00
       }]
     },
     "currentlyProcessing": 14,
     "shippedToday": 20
}

Given a resource like this, I have items (which may be resources or not, embedded or not), and I can tell there's a next page in the collection by the presence of a next link.

I might want to do something with every item in the collection - say compute the average total or log all the totals to a report for the example doc above. I hadn't considered the find or filter operation that @travi mentioned (if I understood correctly) but I imagine you'd need to still check each item and page forward the same way.

So I'm wondering what a great client API would look like - I think ideally the client could totally ignore pagination - it can just iterate over items and paging happens if needed, maybe something like a forEach, map, reduce?

[basti1302]

Oh, I didn't know that using embedded docs for the collection pattern was officially endorsed by the spec. So yeah, let's discuss APIs. I think there are two only loosely related proposals here in this thread:

1. If the client is looking for one specific item, say, specified by a secondary key (something like ea:orders[status:processed], assuming for a moment that status:processed is unique) then Traverson should follow next links automatically if this specific item is not in the current list of embedded orders. I agree that this is a valid feature. If we go in this direction, the $all meta-selector probably should also support paginated collections with next.
2. Offering special operations for collections, like forEach, map, reduce. Now this one is much bigger and also it probably needs quite a bit more discussion. We already have the $all meta selector to get the full array of embedded docs, so you could do something like forEach yourself. I think map, reduce and similar things are outside of the scope of Traverson. Maybe related: There is some discussion about following multiple links (forking the traversal so to say) at How to follow multiple relations from a resource? traverson#7.

[trombka]

Hi,

Given the following resource, where the number of orders is unknown before retrieval, how can I get and iterate over all the linked customer resources? Is it possible with Traverson at all without dealing with _links myself?

{
  "_links": {
    "self": { "href": "http://sample.org/orders" }
  },
  "_embedded": {
    "orders": [
      {
        "_links": {
          "self": { "href": "http://sample.org/orders/1" },
          "customer": { "href": "http://sample.org/customers/91" }
        }
      },
      {
        "_links": {
          "self": { "href": "http://sample.org/orders/2" },
          "customer": { "href": "http://sample.org/customers/92" }
        }
      }      
    ]
  }
}

[basti1302]

@trombka Glad, you asked - yes, you can! Use orders[$all] to retrieve an array of all embedded orders. See https://github.com/basti1302/traverson-hal#embedded-documents

Wait, I misunderstood you, I think. You want an array of the resources at http://sample.org/orders/1, http://sample.org/orders/2, ..., so, in effect, you want traverson to make all these requests and then come back to you with an array of the collected results. This is currently not possible, I'm afraid. It would be an interesting feature, too, I think.

[trombka]

@basti1302 Of course it would be nice to have such collection deep traversal supported by the library, but I think it would good enough to somehow continue traversal from each collection item. A method to start traversal from a resource would do, I think.

traverson
  .from('http://sample.org/orders')
  .follow('orders[$all]')
  .getResource(function (error, orders) {
    orders.forEach(function (order) {
       traverson.from(order).follow('customer'); // 1
       traverson.from(order._links.self.href).follow('customer'); // 2 
       traverson.from(order._links.customer.href) // 3
    });   
  });

1. is not possible now
2. makes an unnecessary request
3. works, but when you have more relations you want follow, it makes the code look ugly
   and the code must be changed when you embed resources (customer in this case)

[savinov]

+1
It would be nice to iterate over a collection (embedded resource) and continue to follow links of each item.
This feature isn't supported yet, is it?