Kafka integration Enhancement request

1. Add Kafka Header Support in Consume

Current behavior: Consume returns only topic, partition, offset, key, value, timestamp.
Missing: Kafka headers.

Customer request:
Expose headers in the correct Kafka format — an ordered list, allowing duplicate keys, for example:

"headers": [

{ "key": "header1", "value": "value1a" },

{ "key": "header1", "value": "value1b" },

{ "key": "header2", "value": "value2" }

]

Reason:
Headers often carry correlation IDs, routing data, schema versioning, etc. Current output loses important metadata. 

2. Consume Without Subscribe  Consumes ALL Messages

Observed behavior:

If the user does not call Subscribe, the Consume action retrieves all events from the beginning, as if: "fromBeginning = true" had been applied automatically.

Customer concern:
This leads to very large payloads and unexpected behavior.

Proposal:
Engineering should evaluate this default behavior to confirm whether Consume should honor committed offsets or if the current “start from beginning” behavior is intentional.

3. Add a maxMessages Option

Current behavior:
If a topic contains many messages (especially from beginning), Consume returns one massive array.

Customer example:
If millions of events exist, the workflow receives all of them at once.

Customer request:
Add:

maxMessages: number

Benefit:
Enables batching, avoids overwhelming workflows, and aligns with common Kafka client capabilities.

4. Event‑Driven Workflow Trigger for Kafka

Current limitation:
RIA workflows can only run:

• On a schedule
• Via API
• Via Automation Center tile

Customer situation:
To process Kafka events, they must repeatedly poll Kafka on a schedule, causing many workflow runs with no new messages.

Customer request:
Add an event-driven trigger so workflows run only when new Kafka messages arrive.

Additional note:
Customer also mentioned similar benefit for MQTT, IBM MQ, etc.