You see examples of changing the schema.įrom Hive or Impala, you can base a new Iceberg table on a schema in a Parquet file. You learn that the Hive or Impala schema changes when the associated Iceberg table changes. Joins, aggregations, and analytical queries, for example, are supported. You can read Iceberg tables from Impala as you would any table. From Impala, this feature is not supported. From Hive, roll back the table data to the state at an older table snapshot, or to a timestamp. In the event of a problem with your table, you can reset a table to a good state as long as the snapshot of the good table is available. For example, you can use projections, joins, filters, and so on. You learn supported transformations and see examples of how to partition a table.įrom Hive, you can query Iceberg metadata tables as you would query a Hive table. By setting a partition spec for an identity transformation partition, you alter the table.įrom Hive or Impala, you can create a table using identity partitioning in which every value is a single partition, or the partition is calculated from values using transformations. To evolve an Iceberg partition from Hive or Impala, you learn to use ALTER to change identity partitions. You set the storage_handler table property to the Iceberg storage handler.Įvolving a partition means changing it without rewriting data files. You can perform actions on an Iceberg table based on the results of a join with a v2 Iceberg table.ĬDP supports table migration from Hive tables to Iceberg tables using ALTER TABLE to set the table properties. Updates to materialized view contents when new data is added to the underlying table are critical otherwise, queries can return stale data. You can create a materialized view of an Iceberg V1 or V2 table based on an existing Hive or Iceberg table. Creating a materialized view on top of Iceberg tables in CDP can further accelerate the performance. ![]() Using a materialized view can accelerate query execution. There is no difference in the way you insert data into a partitioned or unpartitioned Iceberg table. You might need to set the mem_limit or pool configuration (max-query-mem-limit, min-query-mem-limit) to accommodate the load. You learn how to remove snapshots you no longer need.įrom Hive and Impala, you can insert data into Iceberg tables using the standard INSERT INTO a single table.įrom Impala, you can load Parquet or ORC data from a file in a directory on your file system or object store into an Iceberg table. During the lifetime of a table the number of snapshots of the table accumulate. You can expire snapshots that Iceberg generates when you create or modify a table. The syntax you use to create the table determines the default behavior when you drop the Iceberg table from Hive or Impala. ![]() You can use certain Hive and Impala show and describe commands to get information about table metadata. The table properties of the original table are carried over to the new table definition. You see an example of how to use PARTITIONED BY and TBLPROPERTIES to declare the partition spec and table properties for the new table.įrom Hive, you can create an Iceberg table schema based on another table. You can create a partitioned Iceberg table by selecting another table. Create partitioned table as select feature.You can create an Iceberg table based on an existing Hive or Impala table. You learn the subtle differences in these features for creating Iceberg tables from Hive and Impala. You use CREATE TABLE from Impala or CREATE EXTERNAL TABLE from Hive to create an external table in Iceberg. From Hive, you can alter the metadata location of the table if the new metadata does not belong to another table otherwise, an exception occurs. From Impala, you can use ALTER TABLE to rename a table, to change the table owner, or to change the role of the table owner. In Hive or Impala, you can use ALTER TABLE to set table properties. Show many examples of how to run queries on Iceberg tables from Hive and Impala. You can create a view of any table from Hive or Impala using SQL syntax. For example, creating a view is not covered, but Possible query supported from Hive and Impala. This documentation does not attempt to show every Most Hive queries are table-format agnostic. You can run nested, correlated, analytic queries on all supported For example, Impala options are supported in queries Impala queries are table-format agnostic. After meeting prerequisites, you follow step-by-step instructions to use Iceberg features include security and governance, and other Cloudera Data Platform benefits,ĭescribed in Apache Iceberg in CDP. ![]() You can quickly build on your past experience with SQL to analyze Iceberg
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |