Overview
SymmetricDS 3.16 release includes 10 features, 52 improvements, and 55 bug fixes.
Security Fixes
| Issue | Summary | Severity |
|---|---|---|
Prevent logging of secrets in symadmin and wrapper |
Medium |
|
Vulnerability CVE-2025-7962 Severity: MEDIUM Package: org.eclipse.angus:smtp |
Medium |
|
Better input sanitization |
Medium |
Performance Fixes
| Issue | Summary | Severity |
|---|---|---|
Use source staging for incoming batch when engines are hosted together |
Medium |
|
Save column references as numeric values for faster look-up in AbstractDatabaseWriter.getRowData() |
Medium |
|
Ready queues for more efficient push/pulls |
Medium |
|
Multi-threaded routing by channel enabled by default |
Medium |
|
Allow other threads to access the node channel cache while it’s being refreshed |
Medium |
|
Add index on load_id to outgoing_batch table to prevent table scans |
Medium |
What’s New
Channel Improvements
Internal operational SYM tables, such as node, node_security, and incoming/outgoing_error, now use a "system" channel so they act independently and are easier to see as batches for operations versus configuration changes (on the "config" channel). All SYM tables are using channels with a "system" queue so they sync on their own thread even when other channels have a backlog. The "reload" channel for initial and partial loads now defaults to the "bulk" data loader with an increased max batch size of 100,000.
Ready Queues
Ready queues is a feature to make efficient use of multiple queues and their associated threads. Instead of each node communication thread querying for ready data, the cache manager now keeps a list of queues by node that are ready with changes to send, reducing the number of queries needed. Push and pull service use the ready queues to efficiently allocate only the threads needed to send or receive data. Since pull service runs on the target, it still contacts each node on the default queue to get a list of ready queues.
Faster Local Data Transfer
When engines are hosted together (by placing the engine files in the same "engines" directory), a batch for a target engine will be accessed from the source engine’s staging area directly instead of copying it, saving the I/O cost of copying the data again. Remote nodes are unaffected and still receive batches over the network as usual.
Technology Upgrades
H2 database version 2.2 is now included. Users with existing H2 1.x databases may need to add ";MODE=LEGACY" to the end of db.url or "set MODE LEGACY" to db.init.sql (the Pro installer will do this automatically). Users with existing H2 2.0 or 2.1 databases need to export their database from the old version and import into the new one (the Pro installer will do this automatically). The server property of server.http.cookies.enabled is now true by default to accept cookies used by load balancers.
Issues
New Features
3.16.1
SYM-6777 - Add free disk space to runtime properties
3.16.2
SYM-6820 - Run compact script on shutdown H2
3.16.7
SYM-7040 - Add support for the sql_variant column type (in MS SQL Server)
Improvements
3.16.0
SYM-6244 - Upgrade H2 library to 2.2
SYM-6429 - Upgrade dependencies to latest versions
SYM-6537 - Data gap detector should refresh cluster lock
SYM-6577 - Enable acceptance of cookies by default for compatibility with load balancers
SYM-6616 - Save column references as numeric values for faster look-up in AbstractDatabaseWriter.getRowData()
SYM-6625 - Update User Guide with Upgrade and Rollback recommendations
SYM-6691 - Skip querying of self-referencing foreign key child levels if constraints are deferred
SYM-6692 - Prevent logging of secrets in symadmin and wrapper
SYM-6695 - Increase the default max batch size for the reload channel
SYM-6734 - Multi-threaded routing by channel enabled by default
3.16.1
SYM-6678 - Support INCLUDE non-key columns for indexes in Postgres
3.16.2
SYM-6705 - DataExtractorService.transferFromStaging should delete staged file if fails to decrypt staging file
SYM-6752 - Capture of DDL changes and delivery to the target should cause sync triggers at the target node after executing the SQL event
SYM-6793 - The symadmin import-config command should provide a better error message when it lacks the permissions to import a CSV file
SYM-6822 - Add sym_node_host_job_stats to snapshot
SYM-6826 - Add a batch.outgoing.tosend.offline.count to runtime-stats in the Support Snapshot
SYM-6850 - Extract error should be recorded on outgoing batch
SYM-6854 - Add console_table_stats to snapshot
SYM-6865 - Run sync-triggers on SQL events for Create/Alter DDL changes
SYM-6866 - Upgrade gradle to 8.14
3.16.3
SYM-6855 - Acknowledge the registration batch and record it in sym_registration_request
SYM-6862 - Remove trailing slash from sync URL
SYM-6869 - Log a message when the outgoing.batches.max.to.select parameter limits the number of batches selected
SYM-6871 - Skip purging inactive trigger histories still referenced by sym_data (Purge Incoming job)
SYM-6873 - Update the Server Migration section of the User Guide to mention that the db directory must be copied to the new server
SYM-6884 - Snapshot export config as separate CSV
SYM-6893 - On import of config or auto.config.registration.svr.sql.script, make sure node group for current node exists
SYM-6911 - Add engine and log variables to Java router
3.16.4
SYM-6933 - Add number of entries in sym_file_snapshot to support snapshot
SYM-6936 - In the snapshot, move the system batches from outgoing_batch_ok.csv and incoming_batch_ok.csv into separate files
SYM-6947 - Map the MySQL INT UNSIGNED data type to BIGINT
SYM-6953 - Support capture for SAP Hana
SYM-6956 - Sybase alter table locking not called at startup
SYM-6957 - Add a unit test for HttpTransportManager, especially sendAcknowledgement()
SYM-6976 - Push back off max bytes to sync when missing acks
SYM-6988 - Add extension point to handle HTTP connections
3.16.5
SYM-1461 - dbexport --sql column filter
SYM-6971 - Preserve one week of stats in the sym_console_table_stats and sym_node_host_stats tables
SYM-6975 - Set up gitattributes for Git consistently update line endings for Windows and other file systems
SYM-6980 - When the registration server starts up, it should set its own sym_node_security row’s registration_enabled flag to 0 if it’s 1
SYM-6982 - Create unit test for MySqlDdlReader.DetermineExtraColumnInfo
SYM-6990 - Detect when PostgreSQL version supports CreateOrReplace command and set dbInfo.TriggersCreateOrReplaceSupported property
SYM-7000 - Cache the lists of table names in each catalog/schema in the DDL reader
SYM-7008 - Allow other threads to access the node channel cache while it’s being refreshed
3.16.6
SYM-7013 - Implement CODEOWNERS
3.16.8
SYM-7100 - Speed up the sending of deferred constraints and indexes
SYM-7107 - Support snapshot should group captured and parsed batch files in a sub-folder
SYM-7119 - Better input sanitization
SYM-7126 - Add logging when a column name comes back as null after reading metadata
SYM-7144 - Add index on load_id to outgoing_batch table to prevent table scans
SYM-7149 - SQL Server allow config to not query for compressed, filtered, and include columns on indexes
SYM-7152 - Limit amount of time the purge job spends on stranded data
Bug Fixes
3.16.2
SYM-6461 - Blob fields and Firebird 4 or 5
SYM-6779 - Postgresql DDL trigger fails when sym_trigger_hist is located in another schema
SYM-6827 - ExtractDataReader support from target platform
SYM-6830 - NPE when processing table renaming transform (without columns)
SYM-6832 - Unknown exceptions from registration not passed to listeners
SYM-6834 - Custom parameters and parameters with no default do not show up in parameters-changed in the snapshot
SYM-6836 - Fail startup if extensions XML doesn’t initialize Spring context
SYM-6838 - Fails to cancel unprocessed load, stuck in loop
SYM-6847 - Postgres money data type error from initial load
SYM-6861 - Upgrade Spring, Swagger, Commons, and Vaadin libraries
3.16.3
SYM-6876 - SQL Server DDL reader can remove too many outer parentheses from a default value
SYM-6877 - SQL Server DDL reader can cause default values to become invalid by attempting to unescape single quotes
SYM-6879 - Postgres Log Mining permissions should check creation/dropping of logical replication slots instead of querying for roles
SYM-6881 - Deadlock from updating common batch stats in clustered environment
SYM-6886 - Null pointer exception when cancelling all loads for node
SYM-6890 - Default the routing.data.reader.use.multiple.queries parameter to false & document that it can cause data to be out of order
SYM-6901 - Init of engine fails to query non-existent sym_parameter table on DB2 for i (AS400)
SYM-6904 - Missing maxFormKeys setting causes "form with too many fields" error
SYM-6910 - Ready queues prevents load from completing
3.16.4
SYM-6944 - Firebird query error when use of reserved word for column alias
SYM-6945 - Add overrides for isBlob and isClob for AseDatabasePlatform
SYM-6955 - When "on update CURRENT_TIMESTAMP" is not the first part of the extra column info it does not get picked up correctly
SYM-6961 - DBCompare throws a NullPointerException when one database is missing a row with a null value for a UNIVARCHAR column
SYM-6963 - DBCompare can throw a NullPointerException when the source table has unitypes and there are more source rows than target rows
SYM-6972 - A NullPointerException can occur during registration if the config channel cannot be found
SYM-6977 - Oracle XMLTYPE columns get incorrectly flagged as generated
SYM-6985 - Sybase ASE Unitype Fixes
3.16.5
SYM-2247 - dbexport oracle date type with "default sysdate" not compatible with other databases
SYM-5074 - CSV to database replication is not working
SYM-6981 - Reload batches from auto resolver get foreign key error
SYM-6987 - Suppress first error when auto-resolving lines in batch (foreign key or deadlock), escalate on second error for the same line
SYM-7001 - Client node left in "registration pending" after registration complete
3.16.6
SYM-7018 - Global transform script was shared across multiple co-hosted engines
SYM-7024 - Fix java.lang.IllegalArgumentException: No enum constant org.jumpmind.db.sql.SqlTemplateSettings.JdbcLobHandling.PLAİN
SYM-7027 - Retrofit 0006621 into 3.16 NVARCHAR(MAX) and VARCHAR(MAX) columns get modified when another column is updated
SYM-7034 - Leading ZWNBSP character gets lost when replicating unitype data from Sybase ASE
SYM-7047 - Update libraries from dependency scans
3.16.7
SYM-7055 - Replace deprecated code after dependency update
SYM-7057 - Error suppression of batch data conflict only for older Postgres
SYM-7061 - The $(sourceExternalId) and $(targetExternalId) variables don’t get replaced when included in a router’s target catalog/schema
SYM-7083 - When sending a table schema containing a LONGVARBINARY column to an H2 2.x node, the table always gets rebuilt
SYM-7084 - Check trigger.capture.ddl.check.trigger.hist parameter at startup
SYM-7086 - Registration fails when a console role exists with a role ID that comes before its parent role ID alphabetically
3.16.8
SYM-7036 - A database platform’s case sensitivity is not taken into consideration when checking whether an index needs modified
SYM-7097 - Route data should handle missing Node entries from the cache
SYM-7098 - Symmetric does not sort by FK references when sending an all all load
SYM-7112 - Vulnerability CVE-2025-7962 Severity: MEDIUM Package: org.eclipse.angus:smtp
SYM-7139 - Ready queues header should be case insensitive
Tables
The following changes were made to the definition of configuration and runtime tables. Table changes are applied to the database automatically using data definition language (DDL) during startup.
New Columns
| Column Name | Description |
|---|---|
|
Thread number within dynamic queue assigned for extraction |
|
Thread number within dynamic queue assigned for loading over push/pull |
|
The number of rows that are bulk loaded in an extract. |
| Column Name | Description |
|---|---|
|
The timestamp when a partial load was started for this node. |
|
The timestamp when a partial load was completed for this node. |
|
A reference to the load_id in outgoing_batch for the last partial load that occurred. |
|
The user that created the partial load. A null value means that the system created the batch. |
| Column Name | Description |
|---|---|
|
Thread number within dynamic queue assigned for loading over push/pull |
| Column Name | Description |
|---|---|
|
The number of rows that were loaded with the bulk loader. |
Modified Tables
-
Added index
idx_ob_load_id(load_id)
Parameters
The following changes were made to add new parameters, modify their default value, modify their description, or remove them from use.
New Parameters
This is the amount of time ready queue entries will be cached before re-reading them from the database. (Default: 5000)
If source and target engine are hosted together, the source will send a "retry" command instead of sending the batch, and the target will use the source’s outgoing staging to access the batch. This also requires the staging to be enabled (stream.to.file.enabled=true). (Default: true)
The number of milliseconds that the initial load extract job can run a thread for processing extracting requests. (Default: 3600000)
Query for compression on tables and indexes, filtered indexes, and include columns. If you don’t have compressed tables or indexes, or filtered indexes, or include columns, or you are not sending table schema to nodes running SQL Server, set this parameter to false to speed up the reading of metadata. (Default: true)
Feature to install a trigger listening for a Truncate Table event and replicate it to the target node(s). Supported on Postgres only. Requires a service restart. Often used in combination with parameter to enable capture of DDL events (trigger.capture.ddl.changes), but is independent of it. (Default: false)
Whether push and pull should use ready queues to limit queries and communication. Ready queues is the list of queues with outgoing batches that are ready to be sent, queried periodically and cached. When enabled, the pull service first pulls the default queue and receives a list of ready queues that should also be pulled, instead of pulling those queues every time. The push service gets the list of ready queues to allocate threads for calling data extractor service. (Default: true)
This is the maximum number of form keys to synchronize in one connect. A value of 0 or below won’t limit the number of form keys. (Default: 100000)
Modified Parameters
If tables are created as part of the initial load, it will defer setting up table-level logging to improve performance. Applies to loads only. After data is loaded, the table-level logging will be switched on. This parameter needs set for the node that will send the initial load, not the node receiving it. Support for table-level logging varies by database dialect and is ignored where not applicable. Currently this is only supported by the PostgreSQL UNLOGGED feature. Oracle’s NOLOGGING feature might be implemented in the future. Note: this parameter is not compatible with PostgreSQL tables referenced by foreign keys. (Old Default: true) (New Default: false)
For a node operating in offline mode, specify the local directory where data files should be read from. The $(nodeGroupId) and $(externalId) variables are useful when running multiple engines in the same server. (Old Default: tmp/$(nodeGroupId)-$(nodeId)/offline/incoming) (New Default: tmp/$(nodeGroupId)-$(externalId)/offline/incoming)
For a node operating in offline mode, specify the local directory where data files should be written to. The $(nodeGroupId) and $(externalId) variables are useful when running multiple engines in the same server. (Old Default: tmp/$(nodeGroupId)-$(nodeId)/offline/outgoing) (New Default: tmp/$(nodeGroupId)-$(externalId)/offline/outgoing)
When sending an outgoing batch, copy directly from the outgoing staging to the incoming staging when both nodes are on the same server. This also requires the staging to be enabled (stream.to.file.enabled=true). The HTTP transport is still used to send a batch "retry" instruction that causes the target node to read from staging. Deprecated and replaced by incoming.batches.use.source.staging. (Old Default: true) (New Default: false)
Routing reader may run multiple queries for data, with each query including the maximum number of data gaps as specified by the routing.max.gaps.to.qualify.in.sql parameter. This method attempts to use the table’s index for quick results and avoid wasting time on filtering rows that were already routed (as used by the greater.than.query method). If your system is sensitive to data arriving out of order, then this parameter should be set to false. (Old Default: true) (New Default: false)
When enabled, use a thread per channel for parallel routing. (Old Default: false) (New Default: true)