VMware Tanzu Greenplum Community Blog

Ingesting data from Kafka/Rabbit MQ into Greenplum often feels straightforward until you hit a real-world "catch" in the payload format. Recently, I spent significant time solving a challenge involving bank settlement data where a single Kafka message contained a JSON array of hundreds of transactions. Additionally, compliance rules dictated that raw account numbers could never be stored on disk—they had to be hashed immediately. While many developers default to adding a preprocessing microservice (like Kafka Streams or a Python consumer), Greenplum Streaming Server (GPSS) can handle this directly using UDF input transformers . The Core Strategy: ...

In today’s data driven world, scalable and flexible data infrastructure holds prior importance. Greenplum being a powerful and analytical MPP database has become a popular choice when it comes to large scale data processing. In the contemporary data-driven landscape, the establishment of scalable and flexible data infrastructure is of paramount importance. Greenplum, recognized as a robust and analytical Massively Parallel Processing (MPP) database, has emerged as a favored option for the processing of large-scale data. Currently, the installation and operation of a Greenplum cluster mandate the creation of multiple virtual machines, each pre-loaded with ...

Greenplum Database, built on PostgreSQL and designed for massively parallel data processing, powers analytics at scale. As with any large-scale distributed system, operability — the ability to observe, manage, and troubleshoot the system — is critical for long-term maintainability, ensuring uptime, and keeping the system healthy. To streamline operational workflows, Greenplum introduces a Task Scheduler — a lightweight, internal framework designed to run service-oriented tasks at scheduled intervals. Use cases range from scheduled execution of recurring tasks—such as running utilities like gprecoverseg to recover or rebalance segments, or performing ...