[DONE] Errant Triggers causing bulk actions July 24, 2024

Did anyone else experience some strange automation actions happening on July 24, 2024 (starting around 5pm Central US time)?

We have flows that have been in place for months with no issues. Suddenly yesterday one flow caused major unexpected issues however.

The flow is one that is only triggered when a record is updated, filtered for when the item status is changed to = Complete.

Each example shows that this did happen and the flow executed as expected several weeks/months ago. Then suddenly on July 24th it started running in what seemed like an eternal loop even though no edits had occurred recently.

some examples:

Even pausing the culprit flow had no effect. My client ultimately deleted steps of the flow which finally caused the unwanted looping action to stop.

I’ll send a deeper breakdown with specific private details directly to the Tape Team for deeper review. But I’m curious if anyone else experienced a similar issue in the system on this date?

Hi @CarsonRedCliffLabs,

thank you very much for reporting this issue!
We have actually never had a similar issue before, at first I thought that someone maybe had updated a calculation field in this app. Then the update trigger would trigger on multiple records because of the recalculation of the calculation field. However, a filter was set on the status field, which excludes this cause.

We also need to understand the second problem. Simply pausing an automation ensures that the trigger no longer fires, but the existing jobs are still processed. However, if a user has triggered a cacelation of existing jobs, we need to understand why this has not led to a cancellation. We have all the logs and timestamps and can therefore trace them exactly.

We will analyze the issue in detail to understand the root cause and provide you with a detailed analysis and of course fix the identified cause.

Best regards and sorry for the issue
Leo

1 Like

Thanks so much @Leo.
It continues to blow me away that even when we experience an issue with the Tape platform, the Tape team ALWAYS responds quickly and professionally. I end up loving Tape even more rather than getting frustrated. Quite a different and refreshing experience compared to the alternative platform that I use with many other clients…

1 Like

HUGE thank you to @Tim and the Tape Developers for jumping in and doing an in depth review of this situation.
They have communicated quickly and clearly with us directly to resolve the issue.
The end result was discovering this issue happened due to an unexpected scenario which was actually tied to user error, not Tape level server errors.

One filter was slightly incorrect. Then a bulk import had occurred which caused some calculation fields to update. Due to the incorrect filter, and the fact that calc field updates in Tape DO trigger automations (which is different than Podio) - the import caused the flow to trigger and start executing. The newly created items from that flow then caused more calc field updates, which then started the loop issue here.

4 Likes