From 027115a7d827c6c464b5dc304bae128ba347a3e7 Mon Sep 17 00:00:00 2001 From: bubriks Date: Fri, 1 Nov 2024 12:24:37 +0200 Subject: [PATCH 1/3] init --- docs/user_guides/fs/feature_group/create.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/docs/user_guides/fs/feature_group/create.md b/docs/user_guides/fs/feature_group/create.md index 10934f5be..79215e97c 100644 --- a/docs/user_guides/fs/feature_group/create.md +++ b/docs/user_guides/fs/feature_group/create.md @@ -89,6 +89,23 @@ When you create a feature group, you can specify the table format you want to us During the creation of a feature group, it is possible to define the `storage_connector` parameter, this allows for management of offline data in the desired table format outside the Hopsworks cluster. Currently, only [S3](../storage_connector/creation/s3.md) connectors and "DELTA" `time_travel_format` format is supported. +##### Online Config + +While defining online-enabled feature groups it is also possible to configure an online table. You can specify [table options](https://docs.rondb.com/table_options/#table-options) by providing comments. Additionally, it is also possible to define whether online data is stored in memory or on disk using [table space](https://docs.rondb.com/disk_columns/#disk-columns). + +The code example shows the creation of an online-enabled feature group that stores online data on disk using `ts_1` table space and setting several table properties in the comment section. + +``` +fg = fs.create_feature_group( + name='air_quality', + description='Air Quality characteristics of each day', + version=1, + primary_key=['city','date'], + online_enabled=True, + online_config={'table_space': 'ts_1', 'online_comments': ['NDB_TABLE=READ_BACKUP=1', 'NDB_TABLE=PARTITION_BALANCE=FOR_RP_BY_LDM_X_2']} +) +``` + #### Streaming Write API From e18440df0b2f9fecf300389de6a9192307814634 Mon Sep 17 00:00:00 2001 From: bubriks Date: Fri, 1 Nov 2024 12:26:45 +0200 Subject: [PATCH 2/3] mini change --- docs/user_guides/fs/feature_group/create.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user_guides/fs/feature_group/create.md b/docs/user_guides/fs/feature_group/create.md index 79215e97c..0810877e8 100644 --- a/docs/user_guides/fs/feature_group/create.md +++ b/docs/user_guides/fs/feature_group/create.md @@ -89,7 +89,7 @@ When you create a feature group, you can specify the table format you want to us During the creation of a feature group, it is possible to define the `storage_connector` parameter, this allows for management of offline data in the desired table format outside the Hopsworks cluster. Currently, only [S3](../storage_connector/creation/s3.md) connectors and "DELTA" `time_travel_format` format is supported. -##### Online Config +##### Online config While defining online-enabled feature groups it is also possible to configure an online table. You can specify [table options](https://docs.rondb.com/table_options/#table-options) by providing comments. Additionally, it is also possible to define whether online data is stored in memory or on disk using [table space](https://docs.rondb.com/disk_columns/#disk-columns). From 3cea24678db1309c1976a035e41a81395c7f6590 Mon Sep 17 00:00:00 2001 From: Fabio Buso Date: Sun, 10 Nov 2024 09:13:30 +0100 Subject: [PATCH 3/3] Add reference to how to setup table space using the helm charts --- docs/user_guides/fs/feature_group/create.md | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/docs/user_guides/fs/feature_group/create.md b/docs/user_guides/fs/feature_group/create.md index 0810877e8..e1404051b 100644 --- a/docs/user_guides/fs/feature_group/create.md +++ b/docs/user_guides/fs/feature_group/create.md @@ -89,11 +89,11 @@ When you create a feature group, you can specify the table format you want to us During the creation of a feature group, it is possible to define the `storage_connector` parameter, this allows for management of offline data in the desired table format outside the Hopsworks cluster. Currently, only [S3](../storage_connector/creation/s3.md) connectors and "DELTA" `time_travel_format` format is supported. -##### Online config +##### Online Table Configuration -While defining online-enabled feature groups it is also possible to configure an online table. You can specify [table options](https://docs.rondb.com/table_options/#table-options) by providing comments. Additionally, it is also possible to define whether online data is stored in memory or on disk using [table space](https://docs.rondb.com/disk_columns/#disk-columns). +When defining online-enabled feature groups it is also possible to configure the online table. You can specify [table options](https://docs.rondb.com/table_options/#table-options) by providing comments. Additionally, it is also possible to define whether online data is stored in memory or on disk using [table space](https://docs.rondb.com/disk_columns/#disk-columns). -The code example shows the creation of an online-enabled feature group that stores online data on disk using `ts_1` table space and setting several table properties in the comment section. +The code example shows the creation of an online-enabled feature group that stores online data on disk using `ts_1` table space and sets several table properties in the comment section. ``` fg = fs.create_feature_group( @@ -106,6 +106,18 @@ fg = fs.create_feature_group( ) ``` +!!! note Table Space + The table space needs to be provisioned at system level before it can be used. You can do so by adding the following parameters to the values.yaml file used for your deployment with the Helm Charts: + + ```yaml + rondb: + resources: + requests: + storage: + diskColumnGiB: 2 + ``` + + #### Streaming Write API