Skip to Content
Course content

Project 3: Architecting for Scale with Partition Pruning and FDW

Project 3: Architecting for Scale with Partition Pruning and FDW

You are the lead database architect for an IoT company that generates millions of sensor readings daily. Querying this massive, monolithic table has become unacceptably slow.

Your task is to implement declarative table partitioning by range (e.g., partitioning by month) to physically divide the data while maintaining a single logical table interface. Once partitioned, you must simulate a "hot/cold" storage tier architecture. You will spin up a second PostgreSQL instance, configure postgres_fdw (Foreign Data Wrapper), and move all partitions older than one year to this remote "cold" server.

You must write a series of time-series queries spanning both recent and historical data. Using EXPLAIN, you must prove two things: first, that the query planner is successfully executing "Partition Pruning" (skipping irrelevant local partitions), and second, that it is successfully pushing down the remote queries to the FDW rather than pulling the entire remote dataset into local memory to filter.

Rubric:
CriteriaExcellentProficientNeeds Improvement
Partitioning ImplementationFlawlessly implements range partitioning; data routes correctly; handles edge cases and default partitions.Implements partitioning but misses a default partition or uses an inefficient partition key.Fails to successfully partition the data or breaks the logical schema structure.
FDW ConfigurationSuccessfully establishes the foreign server, user mappings, and remote tables with secure and correct configurations.Establishes the FDW but struggles with user mappings or remote execution permissions.Fails to connect the two instances or successfully move the historical data.
Partition Pruning ProofProvides conclusive EXPLAIN output proving the planner skips irrelevant local partitions for bounded queries.Shows an EXPLAIN plan but the planner is still scanning more partitions than strictly necessary.Planner performs full sequential scans across all local partitions.
Query Pushdown VerificationProves via EXPLAIN VERBOSE that WHERE clauses and aggregates are pushed down to the remote server.Proves basic FDW connection but pulls raw data locally to perform filtering or aggregation.Does not understand or demonstrate query pushdown mechanics.
Rating
0 0

There are no comments for now.

to be the first to leave a comment.