.. _radosgw-config-ref:

======================================
 Ceph Object Gateway Config Reference
======================================

The following settings may added to the Ceph configuration file (i.e., usually
``ceph.conf``) under the ``[client.radosgw.{instance-name}]`` section. The
settings may contain default values. If you do not specify each setting in the
Ceph configuration file, the default value will be set automatically.

Configuration variables set under the ``[client.radosgw.{instance-name}]``
section will not apply to rgw or radosgw-admin commands without an instance-name
specified in the command. Thus variables meant to be applied to all RGW
instances or all radosgw-admin options can be put into the ``[global]`` or the
``[client]`` section to avoid specifying ``instance-name``.

.. confval:: rgw_frontends
.. confval:: rgw_data
.. confval:: rgw_enable_apis
.. confval:: rgw_cache_enabled
.. confval:: rgw_cache_lru_size
.. confval:: rgw_dns_name
.. confval:: rgw_script_uri
.. confval:: rgw_request_uri
.. confval:: rgw_print_continue
.. confval:: rgw_remote_addr_param
.. confval:: rgw_op_thread_timeout
.. confval:: rgw_op_thread_suicide_timeout
.. confval:: rgw_thread_pool_size
.. confval:: rgw_num_control_oids
.. confval:: rgw_init_timeout
.. confval:: rgw_mime_types_file
.. confval:: rgw_s3_success_create_obj_status
.. confval:: rgw_resolve_cname
.. confval:: rgw_obj_stripe_size
.. confval:: rgw_extended_http_attrs
.. confval:: rgw_exit_timeout_secs
.. confval:: rgw_get_obj_window_size
.. confval:: rgw_get_obj_max_req_size
.. confval:: rgw_multipart_min_part_size
.. confval:: rgw_relaxed_s3_bucket_names
.. confval:: rgw_list_buckets_max_chunk
.. confval:: rgw_override_bucket_index_max_shards
.. confval:: rgw_curl_wait_timeout_ms
.. confval:: rgw_copy_obj_progress
.. confval:: rgw_copy_obj_progress_every_bytes
.. confval:: rgw_max_copy_obj_concurrent_io
.. confval:: rgw_admin_entry
.. confval:: rgw_content_length_compat
.. confval:: rgw_bucket_quota_ttl
.. confval:: rgw_user_quota_bucket_sync_interval
.. confval:: rgw_user_quota_sync_interval
.. confval:: rgw_bucket_default_quota_max_objects
.. confval:: rgw_bucket_default_quota_max_size
.. confval:: rgw_user_default_quota_max_objects
.. confval:: rgw_user_default_quota_max_size
.. confval:: rgw_account_default_quota_max_objects
.. confval:: rgw_account_default_quota_max_size
.. confval:: rgw_verify_ssl
.. confval:: rgw_max_chunk_size

Lifecycle Settings
==================

Bucket Lifecycle (LC) configuration can be used to manage your objects so that
they are stored effectively throughout their lifetimes. In past releases,
lifecycle processing was rate-limited by single-threaded processing. As of the
Nautilus release, the Ceph Object Gateway allows for parallel-thread processing
of bucket lifecycles across additional Ceph Object Gateway instances and
replaces in-order index-shard enumeration with a random ordered sequence.

Two options in particular are relevant to adjusting the aggressiveness of
lifecycle processing:

.. confval:: rgw_lc_max_worker
.. confval:: rgw_lc_max_wp_worker

These values can be tuned based upon your specific workload to further increase
the aggressiveness of lifecycle processing. For a workload with a large number
of buckets (thousands), raise the number of workers by increasing
:confval:`rgw_lc_max_worker` from the default value of 3. But for a workload
with a higher number of objects per bucket (hundreds of thousands), raise the
number of parallel threads by increasing :confval:`rgw_lc_max_wp_worker` from
the default value of 3.

.. note:: Before increasing either of these values, validate the current
   Cluster performance and Ceph Object Gateway utilization.

The lifecycle maintenance thread must also be enabled on at least one RGW
daemon for each zone. 

.. confval:: rgw_enable_lc_threads

Garbage Collection Settings
===========================

The Ceph Object Gateway allocates storage for new objects immediately.

The Ceph Object Gateway purges the storage space used for deleted and overwritten 
objects in the Ceph Storage cluster some time after the gateway deletes the 
objects from the bucket index. The process of purging the deleted object data 
from the Ceph Storage cluster is known as Garbage Collection or GC.

To view the queue of objects awaiting garbage collection, execute the following

.. prompt:: bash #

   radosgw-admin gc list

.. note:: Specify ``--include-all`` to list all entries, including unexpired
   Garbage Collection objects.

Garbage collection is a background activity that may
execute continuously or during times of low loads, depending upon how the
administrator configures the Ceph Object Gateway. By default, the Ceph Object
Gateway conducts GC operations continuously. Since GC operations are a normal
part of Ceph Object Gateway operations, especially with object delete
operations, objects eligible for garbage collection exist most of the time.

Some workloads may temporarily or permanently outpace the rate of garbage
collection activity. This is especially true of delete-heavy workloads, where
many objects get stored for a short period of time and then deleted. For these
types of workloads, administrators can increase the priority of garbage
collection operations relative to other operations with the following
configuration parameters.

.. confval:: rgw_gc_max_objs
.. confval:: rgw_gc_obj_min_wait
.. confval:: rgw_gc_processor_max_time
.. confval:: rgw_gc_processor_period
.. confval:: rgw_gc_max_concurrent_io

:Tuning Garbage Collection for Delete Heavy Workloads:

As an initial step towards tuning Ceph Garbage Collection to be more
aggressive the following options are suggested to be increased from their
default configuration values::

  rgw_gc_max_concurrent_io = 20
  rgw_gc_max_trim_chunk = 64

.. note:: Modifying these values requires a restart of the RGW service.

Once these values have been increased from default please monitor for performance of the cluster during Garbage Collection to verify no adverse performance issues due to the increased values.

At least one RGW in each zone must have the garbage collection maintenance
thread running:

.. confval:: rgw_enable_gc_threads

Multisite Settings
==================

.. versionadded:: Jewel

You may include the following settings in your Ceph configuration
file under each ``[client.radosgw.{instance-name}]`` instance.

.. confval:: rgw_zone
.. confval:: rgw_zonegroup
.. confval:: rgw_realm
.. confval:: rgw_run_sync_thread
.. confval:: rgw_data_log_window
.. confval:: rgw_data_log_changes_size
.. confval:: rgw_data_log_num_shards
.. confval:: rgw_md_log_max_shards
.. confval:: rgw_data_sync_poll_interval
.. confval:: rgw_meta_sync_poll_interval
.. confval:: rgw_bucket_sync_spawn_window
.. confval:: rgw_data_sync_spawn_window
.. confval:: rgw_meta_sync_spawn_window

.. important:: The values of :confval:`rgw_data_log_num_shards` and
   :confval:`rgw_md_log_max_shards` should not be changed after sync has
   started.

S3 Settings
===========

.. confval:: rgw_s3_auth_use_ldap

Swift Settings
==============

.. confval:: rgw_enforce_swift_acls
.. confval:: rgw_swift_tenant_name
.. confval:: rgw_swift_token_expiration
.. confval:: rgw_swift_url
.. confval:: rgw_swift_url_prefix
.. confval:: rgw_swift_auth_url
.. confval:: rgw_swift_auth_entry
.. confval:: rgw_swift_account_in_url
.. confval:: rgw_swift_versioning_enabled
.. confval:: rgw_trust_forwarded_https

Logging Settings
================

.. confval:: rgw_log_nonexistent_bucket
.. confval:: rgw_log_object_name
.. confval:: rgw_log_object_name_utc
.. confval:: rgw_usage_max_shards
.. confval:: rgw_usage_max_user_shards
.. confval:: rgw_enable_ops_log
.. confval:: rgw_enable_usage_log
.. confval:: rgw_ops_log_rados
.. confval:: rgw_ops_log_socket_path
.. confval:: rgw_ops_log_data_backlog
.. confval:: rgw_usage_log_flush_threshold
.. confval:: rgw_usage_log_tick_interval
.. confval:: rgw_log_http_headers

Keystone Settings
=================

.. confval:: rgw_keystone_url
.. confval:: rgw_keystone_admin_domain
.. confval:: rgw_keystone_admin_project
.. confval:: rgw_keystone_admin_token
.. confval:: rgw_keystone_admin_token_path
.. confval:: rgw_keystone_admin_tenant
.. confval:: rgw_keystone_admin_user
.. confval:: rgw_keystone_admin_password
.. confval:: rgw_keystone_admin_password_path
.. confval:: rgw_keystone_accepted_roles
.. confval:: rgw_keystone_token_cache_size
.. confval:: rgw_keystone_verify_ssl
.. confval:: rgw_keystone_service_token_enabled
.. confval:: rgw_keystone_service_token_accepted_roles
.. confval:: rgw_keystone_expired_token_cache_expiration

Server-side Encryption Settings
===============================

.. confval:: rgw_crypt_s3_kms_backend

Barbican Settings
=================

.. confval:: rgw_barbican_url
.. confval:: rgw_keystone_barbican_user
.. confval:: rgw_keystone_barbican_password
.. confval:: rgw_keystone_barbican_tenant
.. confval:: rgw_keystone_barbican_project
.. confval:: rgw_keystone_barbican_domain

HashiCorp Vault Settings
========================

.. confval:: rgw_crypt_vault_auth
.. confval:: rgw_crypt_vault_token_file
.. confval:: rgw_crypt_vault_addr
.. confval:: rgw_crypt_vault_prefix
.. confval:: rgw_crypt_vault_secret_engine
.. confval:: rgw_crypt_vault_namespace

SSE-S3 Settings
===============

.. confval:: rgw_crypt_sse_s3_backend
.. confval:: rgw_crypt_sse_s3_vault_secret_engine
.. confval:: rgw_crypt_sse_s3_key_template
.. confval:: rgw_crypt_sse_s3_vault_auth
.. confval:: rgw_crypt_sse_s3_vault_token_file
.. confval:: rgw_crypt_sse_s3_vault_addr
.. confval:: rgw_crypt_sse_s3_vault_prefix
.. confval:: rgw_crypt_sse_s3_vault_namespace
.. confval:: rgw_crypt_sse_s3_vault_verify_ssl
.. confval:: rgw_crypt_sse_s3_vault_ssl_cacert
.. confval:: rgw_crypt_sse_s3_vault_ssl_clientcert
.. confval:: rgw_crypt_sse_s3_vault_ssl_clientkey


QoS Settings
============

.. versionadded:: Nautilus

The older and now non-default ``civetweb`` frontend has a threading model that uses a thread per
connection and hence is automatically throttled by :confval:`rgw_thread_pool_size`
when accepting connections. The newer and default ``beast`` frontend is
not limited by the thread pool size when it comes to accepting new
connections, so a scheduler abstraction was introduced in the Nautilus release
to support additional methods of scheduling requests.

Currently the scheduler defaults to a throttler that limits active
connections to a configured limit. QoS rate limiting based on mClock is currently
*experimental* phase and not recommended for production. The current
implementation of the *dmclock_client* op queue divides RGW ops into admin, auth
(swift auth, sts) metadata, and data requests.


.. confval:: rgw_max_concurrent_requests
.. confval:: rgw_scheduler_type
.. confval:: rgw_dmclock_auth_res
.. confval:: rgw_dmclock_auth_wgt
.. confval:: rgw_dmclock_auth_lim
.. confval:: rgw_dmclock_admin_res
.. confval:: rgw_dmclock_admin_wgt
.. confval:: rgw_dmclock_admin_lim
.. confval:: rgw_dmclock_data_res
.. confval:: rgw_dmclock_data_wgt
.. confval:: rgw_dmclock_data_lim
.. confval:: rgw_dmclock_metadata_res
.. confval:: rgw_dmclock_metadata_wgt
.. confval:: rgw_dmclock_metadata_lim

.. _Architecture: ../../architecture#data-striping
.. _Pool Configuration: ../../rados/configuration/pool-pg-config-ref/
.. _Cluster Pools: ../../rados/operations/pools
.. _Rados cluster handles: ../../rados/api/librados-intro/#step-2-configuring-a-cluster-handle
.. _Barbican: ../barbican
.. _Encryption: ../encryption
.. _HTTP Frontends: ../frontends

D4N Settings
============

D4N is a caching architecture that utilizes Redis to speed up S3 object storage 
operations by establishing shared databases among Ceph Object Gateway (RGW) daemons.

The D4N architecture can only function on one Redis instance at a time. 
The address is configurable and can be changed by accessing the parameters 
below.

.. confval:: rgw_d4n_address
.. confval:: rgw_d4n_l1_datacache_persistent_path
.. confval:: rgw_d4n_l1_datacache_disk_reserve
.. confval:: rgw_d4n_l1_evict_cache_on_start
.. confval:: rgw_d4n_l1_fadvise
.. confval:: rgw_d4n_libaio_aio_threads
.. confval:: rgw_d4n_libaio_aio_num
.. confval:: rgw_lfuda_sync_frequency
.. confval:: rgw_d4n_l1_datacache_address

Topic Persistency Settings
==========================

Topic persistency will repeatedly push notifications until they succeed.
For more information, see `Bucket Notifications`_.

The default behavior is to push indefinitely and as frequently as possible.
With these settings you can control how long and how often to retry an
unsuccessful notification by configuring the maximum retention time and/or or
maximum number of retries.
The interval between push retries can be configured via the sleep duration
parameter.

All of these options default to the value `0`, which means that persistent
retention is indefinite, and notifications are retried as frequently as possible.

.. confval:: rgw_topic_persistency_time_to_live
.. confval:: rgw_topic_persistency_max_retries
.. confval:: rgw_topic_persistency_sleep_duration

.. _Bucket Notifications: ../notifications
   
Cloud Restore Settings
======================

Cloud Restore feature currently enables the restoration of objects transitioned to S3-compatible cloud services into Ceph Object Gateway (RGW). The restore requests are asynchronously processed by Restore worker thread in the background. 

.. confval:: rgw_restore_max_objs
.. confval:: rgw_restore_lock_max_time
.. confval:: rgw_restore_processor_period

These values can be tuned based upon your specific workload to further increase the aggressiveness of restore processing. 
