[Feature] Accept all task custom fields when running as a Fivetran quickstart transformation

[irvingpop]

Is there an existing feature request for this?

• I have searched the existing issues

Describe the Feature

I would love to run this as a Fivetran quick transformation (in Fivetran), rather than in my dbt pipelines. The only thing preventing me from doing so is the task_pass_through_columns configuration I need.

It would be super cool if this package had some kind of "include all custom task fields" setting. Even better, if that was automatically switched on when running via Fivetran.

Describe alternatives you've considered

No response

Are you interested in contributing this feature?

• Yes.
• Yes, but I will need assistance and will schedule time during your office hours for guidance.
• No.

Anything else?

No response

[fivetran-joemarkiewicz]

Hi @irvingpop thanks for opening this Feature Request!

Variable configurations is something I know the product team is looking to explore in the future. In the meantime, do you know if there is a specific field(s) you are wishing to incorporate? Maybe we can consider permanently adding new fields by default if they aren't too custom. Alternatively, we have leveraged solutions in the past which do include all custom fields by default for all users, but this comes with some tradeoffs (we aren't able to document these fields and ensure they do not cause any downstream issues).

Let me know your thoughts. I'll also share this feedback with our product team to help them understand the desire for the variable configuration ability in the Quickstart product.

[irvingpop]

Thanks for the quick response @fivetran-joemarkiewicz ! Exciting to hear that it's on the radar for your product team 🙂

In my case - our field names are very custom and team/domain-specific. Asana seems to make it easy to go crazy with custom fields, and our various eng teams seem to come up with some hilarious ones (I'm still chuckling at these 3: custom_explodiness, custom_explody, custom_explody_ness 🤣 )

I agree that people who run this module seriously should be recommended to maintain a static list of known and trusted custom fields, and to document them. And it would be super cool to be able to do that within the Fivetran UI! But even if not, or just for "quickstart" purposes, I can see some advantages to a "take all" strategy

[fivetran-joemarkiewicz]

custom_explodiness has to go down in history for a very epic custom field name 😆

Thanks @irvingpop for the additional context and details! Unfortunately, while the variable use case in the Fivetran UI is on the roadmap, I don't believe it is slated for anytime soon. Let me consult with our product team to consider the pros/cons of the "take all" strategy and if this is something we would want to consider. Another risk I see with this is given the fact that Asana allows teams to go crazy with custom fields means that if we apply the take all strategy then the end model can become extremely wide with possibly hundreds of fields, making it hard to effectively leverage.

I know it isn't the best solution, but a possible workaround for the time being could be to perform a post Quickstart join from the asana__tasks table to the source task table to bring in the additional fields. I know this isn't ideal, but could be a solution for the meantime. I'll circle back once I have some details from our product team!

[irvingpop]

perform a post Quickstart join from the asana__tasks table to the source task table to bring in the additional fields

This is a good workaround idea, and I can live with that for now. For the rest - very excited to see what y'all come up with. I could definitely see us running more quickstart transformations in Fivetran in the future if they had a bit of configuration flexibility.

[fivetran-joemarkiewicz]

Sounds great @irvingpop and glad that suggestion will suffice for the time being.

I relayed this with our product team and we are not going to move forward with a "take all" approach by default. The product team is also planning on incorporating the ability to customize the Quickstart data models via these variables later in the year. I will post back here once this functionality is possible directly within Quickstart. For the time being the post transformation join will be the solution for Quickstart users.