FirebirdSQL logo
 New In Firebird 5.0Changes to the Firebird API and ODS 

Support for parallel operations

Vlad Khorsun

Tracker ticket: #7447

The Firebird engine can now execute some tasks using multiple threads in parallel.Currently, parallel execution is implemented for the sweep and the index creation tasks.Parallel execution is supported for both automatic and manual sweep.

To handle a task with multiple threads, the engine runs additional worker threads and creates internal worker attachments.By default, parallel execution is not enabled.There are two ways to enable parallelism in a user attachment:

  1. set the number of parallel workers in DPB using new tag isc_dpb_parallel_workers,

  2. set the default number of parallel workers using new setting ParallelWorkers in firebird.conf.

The gfix utility has a new command-line switch, -parallel, that allows to set the number of parallel workers for the sweep task.

For example, the following will run sweep on the given database and asks the engine to use 4 workers:

gfix -sweep -parallel 4 <database>

gfix uses DPB tag isc_dpb_parallel_workers when attaches to <database>, if switch -parallel is present.

A similar option was also added to gbak.

The new firebird.conf setting ParallelWorkers sets the default number of parallel workers that can be used by any user attachment running parallelizable task.The default value is 1 and means no use of additional parallel workers.The value in the DPB has a higher priority than the setting in firebird.conf.

To control the number of additional workers that can be created by the engine, there are two new settings in firebird.conf:

ParallelWorkers

Sets the default number of parallel workers used by a user attachments.Can be overridden by attachment using tag isc_dpb_parallel_workers in DPB.

MaxParallelWorkers

Limits the maximum number of simultaneously used workers for the given database and Firebird process.

Internal worker attachments are created and managed by the engine itself.The engine maintains per-database pools of worker attachments.The number of threads in each pool is limited by the value of the MaxParallelWorkers setting.The pools are created by each Firebird process independently.

In SuperServer architecture worker attachments are implemented as light-weight system attachments, while in Classic and SuperClassic they look like usual user attachments.All worker attachments are embedded into the creating server process.Thus, in Classic architectures there are no additional server processes.Worker attachments are present in monitoring tables.Idle worker attachments are destroyed after 60 seconds of inactivity.Also, in Classic architectures, worker attachments are destroyed immediately after the last user connection detaches from the database.

Examples:

Set in firebird.conf ParallelWorkers = 4, MaxParallelWorkers = 8 and restart Firebird server.

  1. Connect to test database not using isc_dpb_parallel_workers in DPB and execute a CREATE INDEX …​ SQL statement.On commit, the index will be created and the engine will use three additional worker attachments.In total, four attachments in four threads will work on index creation.

  2. Ensure auto-sweep is enabled for test database.When auto-sweep runs on that database, it will also use three additional workers (and run within four threads).

  3. More than one task at a time can be parallelized: make two attachments and execute a CREATE INDEX …​ in each of them (of course indices to be built should be different).Each index will be created using four attachments (one user and three worker) and four threads.

  4. Run gfix -sweep <database> without specifying switch -parallel: sweep will run using four attachments in four threads.

  5. Run gfix -sweep -parallel 2 <database>: sweep will run using two attachments in two threads.This shows that value in DPB tag isc_dpb_parallel_workers overrides value of setting ParallelWorkers.