Thus, a Pinot server hosts one or more helix partitions of one Segments of a table are modeled as Helix partitions (of a resource). Pinot Servers are modeled as Helix Participants, hosting Pinot tables (referred to as resources in helix terminology). Pinot can use other storage systems such as HDFS or ADLS. Of them are configured with the same back-end storage system so that they have a common view of the segments ( e.g. If there are multiple controllers, Pinot expects that all There can be multiple instances of Pinot controller for redundancy. Pinot Controller hosts Helix Controller, in addition to hosting REST APIs for Pinot cluster administration and data ingestion. In the cluster and ensuring that state constraints are satisfied while maintaining cluster stability It is responsible for coordinating all transitions Controller The node that observes and controls the Participant nodes. Spectators are notified of state changes in the cluster (state of a participant, or that of a partition in a participant).

Helix uses Zookeeper to store cluster state and metadata.īriefly, Helix divides nodes into three logical components based on their responsibilities: Participant The nodes that host distributed, partitioned resources Spectator The nodes that observe the current state of each Participant and use that information to access the resources. Helix is a cluster management framework to manage replicated, partitioned resources in a distributed system. Pinot leverages Apache Helix for cluster management.

Pinot Server Hosts one or more segments and serves queries from those segments Pinot Broker Accepts queries from clients and routes them to one or more servers, and returns consolidated response to the client. Pinot Components ¶ Pinot Controller Manages other pinot components (brokers, servers) as well as controls assignment of tables/segments to servers.

Helix server hosting