SplitByGroup and App Builder question
I am trying to match what I think happens with SplitByGroup and what the App Builder visually displays.
When I have a SplitByGroup all synchronous downstream tasks have separate instances running for each group. The SplitByGroup is terminated once you call an asynchronous task, correct?
In my app below I have a Split followed by a Dwell and a Window all of which are synchronous so part of the SplitByGroup. Then I have a Procedure task which is asynchronous but I still have the blue line which indicates to me that the Split is still active. The second Procedure is back to a thin grey line which visually suggests, to me anyway, that my split doesn’t terminate until the second procedure call.
Also I noticed that a Transform is a synchronous task, why is that? It is just calling a procedure and the procedure call is made by the event stream rather than the rule. For procedure calls the event stream is empty and the rule is executing the procedure.
Thanks.
  • This topic was modified 4 years, 7 months ago by dsprinzen.
  • This topic was modified 4 years, 7 months ago by dsprinzen.
  • This topic was modified 4 years, 5 months ago by Ken.
  • This topic was modified 4 years, 5 months ago by Ken.
  • This topic was modified 4 years, 5 months ago by dsprinzen.
Posted: January 14, 2020 at 6:56 pm
Replies: 3
Jan 15, 2020
Posted by dsprinzen

I think the issue here stems from how you’re thinking of async activities. Dwell and Window are not considered async activities because they don’t have to do any asynchronous work to determine what event to emit to the next downstream task. Dwell might not emit every inbound event to the next downstream task, but it makes that decision synchronously for each event as it arrives.
The tasks that are marked async all have the potential to do database or source operations which we need to wait to complete before we can emit an event to the next downstream task. This includes Procedure execution, Save to Type, Publish to Source, Start Collaboration, and a couple others.

The reason for this distinction has to do with the implementation of Event Streams. Event streams can only process one event at a time (per group if there is an upstream split), so this sort of asynchronous work that we need to wait for can get in the way of the flow of events through the event stream. For these tasks, we do the async work in a rule instead because multiple executions of a rule can occur concurrently. The consequence of this is that when the event hits the rule it effectively hits the end of a chain of event streams. Split By Group persists as long as you remain in a chain of event streams, however as soon as you put a rule in the middle you’ve broken that chain, and the split doesn’t persist after the rule on to the next downstream task.

Now about Transformation: Transformation is intended to be used to do in-memory transformations that do not require expensive operations. It’s possible to abuse this activity pattern by putting expensive operations in your transformations, in which case you won’t get the throughput you want through the transformation task, but we don’t stop you from making that bad decision. If you want to do expensive transformation-like things that require DB operations you can always use the Procedure activity pattern, which as you noted is async.

Jan 16, 2020
Posted by dsprinzen

Hi Jake,

Thanks for the explanation I didn’t get that distinction between sync and async but makes perfect sense.

But why in the App builder is there a solid blue connection between my Window and my Procedure call. If I understood once I hit a Rule (the first proc call) the splitbygroup is over. The App Builder makes it look like the first procedure call is still part of the SlipByGroup.

Jan 16, 2020
Posted by dsprinzen

The split persists through the window on to whatever task is next because window is not async. You’re right that the procedure activity pattern is implemented with a rule, so the split ends there. But the thing linking the upstream window to the downstream procedure call is an event stream, and the split still applies to that event stream, so the line is blue. Imagine if you had 2 tasks downstream from your window like this:
splitby

It would be strange to have a blue line going to filter and a grey line going to Procedure when they are both triggered off of the same event stream (the event stream that implements the Transform task). That event stream that triggers both Procedure and Filter is still affected by the upstream Split, so the blue line is accurate.

  • This reply was modified 4 years, 7 months ago by dsprinzen.
  • This reply was modified 4 years, 3 months ago by dsprinzen.
Viewing 4 posts - 1 through 4 (of 4 total)
  • You must be logged in to reply to this topic.
© Vantiq 2024 All rights reserved  •  Vantiq Corporation’s Privacy Policy and Cookie Policy