firebird/firebird-qa
6
0
Fork 0
mirror of https://github.com/FirebirdSQL/firebird-qa.git synced 2025-01-22 13:33:07 +01:00
Code Issues Activity

Switch from setuptools to hatch, version 0.19.0

Browse Source
This commit is contained in:
Pavel Císař 2024-02-08 17:03:56 +01:00
parent 310b2c2995
commit 70e75e4b56
13 changed files with 825 additions and 347 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

146
CHANGELOG.md Normal file
View File

@ -0,0 +1,146 @@
# Change Log
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/)
and this project adheres to [Semantic Versioning](http://semver.org/).
## [0.19.0] - 2024-02-08
### Changed
- Switch from `setuptools` to `hatch`.
- Updated dependencies.
## [0.18.0] - 2023-02-14
### Added
- Added cache for empty databases. This works transparently and does not require any
special configuration. Databases are stored in `dbcache` subdirectory (created automatically)
for combination of ODS + page size + SQL dialect + character set.
Files in `dbcache` directory could be removed as needed (including whole directory)
to fore creation of new database.
Cache is enabled by default. Use new --disable-db-cache option to disable it.
## [0.17.3] - 2023-02-14
### Added
- Added `--driver-config` option to specify different filename for driver configuration.
## [0.17.2] - 2023-01-17
### Fixed
- Trace session support in plugin now uses service query with timeout (provided by
`firebird-driver 1.8.0`) and terminates the trace thread gracefuly even if terminating
trace session fails.
## [0.17.1] - 2022-11-21
### Added
- When database initialization script fails, the XML output is extended with `dbinit-stderr`
property that contains `stderr` output with errors reported by ISQL.
### Fixed
- Uregistered bug in trace.TraceConfig - redundant `flags` definition.
## [0.17.0] - 2022-06-30
### Added
- Added `Mapping` and `mapping_factory`.
### Changed
- Variable `test_cfg` renamed to `QA_GLOBALS`.
## [0.16.0] - 2022-06-19
### Added
- Added support for configuration of tests. A `configparser.ConfigParser` instance is
available as `test_cfg`. This instance is initialized with values from file `test_config.ini`
located in `files` subdirectory.
## [0.15.2] - 2022-06-13
### Fixed
- Fix problem with database init script. Now it uses the database charset instead default
UTF8. The UTF8 is used only when database charset is not specified.
## [0.15.1] - 2022-06-08
### Added
- Added `encryption` marker to mark test as requiring the encryption plugin
### Changed
- Package `psutil` is now a dependency, installed automatically with plugin.
## [0.15.0] - 2022-06-05
### Added
- Added possibility to use databases aliases. The `db_factory()` parameter `filename` is
now handled as database alias if it starts with `#`, for example `#employee` means alias
`employee`. The alias must be defined in `databases.conf` file.
When filename is an alias, the `Database.db_path` property does not contain
full `pathlib.Path` to the database, but this database alias.
### Changed
- To simplify portable use of databases with special configuration via `databases.conf`,
the plugin initialization now ensures empty subdirectory `QA` in Firebird sample directory.
To define your test databases in `databases.conf`, use next pattern:
```
my_db = $(dir_sampleDB)/QA/my-db.fdb
{
...
}
```
On plugin initialization, the `QA` sub-directory is first emptied and removed, and then
newly created. On non-Windows, full privileges are granted.
## [0.14.0] - 2022-05-12
### Added
- Added possibility to specify user, password and role in `Action.connect_server()` and
`.Action.trace()`
### Changed
- DataList is now generic class.
- DataList.extract() has new 'copy' argument.
## [0.13.1] - 2022-05-12
### Fixed
- Fixed problem with service encoding
- Fixed problem with tags in User
## [0.13.0] - 2022-04-19
### Added
- Explicit `Optional` typing.
- Added support for both `encoding` and `encoding_errors` in `Action.connect_server()`
and `Action.trace()`.
## [0.12.1] - 2022-02-24
Initial release.

159
PLUGIN-README.md Normal file
View File

@ -0,0 +1,159 @@
# pytest plugin for Firebird QA
## Installation
If you plan to use this plugin for personal purposes (not related to Firebird project QA),
we recommend to use `pipx` to install `pytest` together with `firebird-qa` plugin:
```
pipx install pytest
pipx inject pytest firebird-qa
```
## Configuration
### Firebird-driver configuration
The QA plugin uses firebird-driver to access the Firebird servers, and uses driver configuration
object to set up various driver and server/database connection parameters. The configuration object
is initialized from `firebird-driver.conf` file, and plugin specifically utilizes server sections
in this file. When pytest is invoked, you must specify tested server with `–server <name>` option,
where `<name>` is name of server configuration section in `firebird-driver.conf` file.
This file is stored in firebird-qa repository, and defines default configuration suitable to most QA setups.
Note:
The `firebird-driver.conf` file should be located in QA root directory. In default setup, the QA plugin
is used to test local Firebird installation with default user name and password (SYSDBA/masterkey)
via local server (configuration section).
Important:
The firebird-driver currently does not support specification of client library in server sections.
However, the QA plugin works around that limitation. If server section for tested server contains
`fb_client_library` option specification, it’s copied to global setting.
See configuration chapter in [driver documentation](https://firebird-driver.readthedocs.io) for details.
### Pytest configuration
While it’s not required, it’s recommended to create pytest configuration file in QA root directory.
You may use this file to simplify your use of pytest with addopts option, or adjust pytest behaviour.
Suggested options for `pytest.ini`:
```
console_output_style = count
testpaths = tests
addopts = --server local --install-terminal
```
## Running test for Firebird
To run all tests in suite against local Firebird server, invoke:
```
pytest --server local ./tests
```
Tip: If you created `pytest.ini` with recommended values, you can just invoke pytest without additional parameters.
### pytest report header
When pytest is invoked, a report header is printed on terminal before individual tests are executed.
The QA plugin extend this header with next information:
- Python encodings
- system
- locale
- filesystem
- Information about tested Firebird server
- conf. section name
- version
- mode
- architecture
- home directory
- tools directory
- used client library
### pytest switches installed by QA plugin
The QA plugin installs several pytest command-line switches. When you run pytest ``--help``,
they are listed in Firebird QA section:
```
Firebird QA:
--server=SERVER Server configuration name
--bin-dir=PATH Path to directory with Firebird utilities
--protocol={xnet,inet,inet4,wnet}
Network protocol used for database attachments
--runslow Run slow tests
--save-output Save test std[out|err] output to files
--skip-deselected={platform,version,any}
SKIP tests instead deselection
--extend-xml Extend XML JUnit report with additional information
--install-terminal Use our own terminal reporter
```
**server:**
REQUIRED option. Section name in firebird-driver.conf with connection parameters for tested server.
**bin-dir:**
Normally, the QA plugin detects and properly sets the directory where Firebird tools are installed.
However, you can set this directory explicitly using the --bin-dir switch.
**protocol:**
Override for network protocol specified in firebird-driver.conf file (or default).
**runslow:**
Tests that run for longer than 10 minutes on equipment used for regular Firebird QA should be
marked as slow. They are not executed, unless this switch is specified.
**save-output:**
_Experimental switch_
When this switch is specified, stdout/stderr output of external Firebird tool executed by
test is stored in `/out` subdirectory. Intended for test debugging.
**skip-deselected:**
Tests that are not applicable to tested server (because they are for specific platform or
Firebird versions) are deselected during pytest collection phase. It means that they are not
shown in test session report. This switch changes the routine, so tests are marked to skip
(with message explaining why) instead deselection, so they show up is session report.
**extend-xml:**
When this switch is used together with `--junitxml` switch, the produced JUnitXML file will
contain additional metadata for testsuite and testcase elements recorded as property sub-elements.
**Important:**
Please note that using this feature will break schema verifications for the latest JUnitXML schema.
This might be a problem when used with some CI servers.
**install-terminal:**
This option changes default pytest terminal reporter that displays pytest NODE IDs, to custom
reporter that displays Firebord QA test IDs.
pytest node IDs are of the form `module.py::class::method` or `module.py::function`.
Firebord QA test IDs are defined in our test metadata.
**Important:**
Right now, the custom terminal is opt-in feature. This will be changed in some future release
to opt-out using new switch.
### Test for use with this plugin
Please read the [plugin documentation](https://firebird-qa.rtfd.io) for instructions how
to create tests that use special support provided by this plugin.

293
README.md Normal file
View File

@ -0,0 +1,293 @@
# Firebird QA
This package contains:
- pytest plugin that provides support for testing the Firebird engine. It uses new Python
driver for Firebird (`firebird-driver`).
- tests for Firebird engine (directory `tests`)
- files needed by tests (directories `databases`, `files`, `backups` and `configs`)
**Requirements:** Python 3.8+, Firebird 3+
You should definitelly read the [QA suite documentation](https://firebird-qa.readthedocs.io)!
## Quickstart
1. Clone the git repository
2. Install the plugin and required dependencies by running next command from repo. directory::
pip install -e .
3. Adjust Firebird server configuration.
**ONLY FOR MANUAL runs:**
Check content of `$FB_HOME/databases.conf.`
Ensure that RemoteAccess is allowed for `security.db`.
Also, it is recommended to set number of buffers not less than 256 for it:
```
security.db = $(dir_secDb)/security<suffix>.fdb
{
RemoteAccess = true
DefaultDbCachePages = 256
}
```
This must be done only if you want to run some tests manually.
Automated scenario for running tests will overwrite this file
and put there all needed data before every pytest session (using
`$QA_ROOT/files/qa-databases.conf` as prototype for that purpose).
1. `$FB_HOME/firebird.conf`:
**Firebird 3:**
```
# Required
BugcheckAbort = 1
ExternalFileAccess = Full
AuthServer = Srp, Win_Sspi, Legacy_Auth
UserManager = Srp, Legacy_UserManager
WireCrypt = Enabled
FileSystemCacheThreshold = 99999K
IpcName = xnet_fb3x_qa
RemotePipeName = wnet_fb3x_qa
# Recommended
DefaultDbCachePages = 10000
MaxUnflushedWrites = -1
MaxUnflushedWriteTime = -1
# Needed for encryption-related tests.
KeyHolderPlugin = fbSampleKeyHolder
```
**Firebird 4:**
```
# Required
BugcheckAbort = 1
ExternalFileAccess = Full
AuthServer = Srp, Win_Sspi, Legacy_Auth
UserManager = Srp, Legacy_UserManager
ReadConsistency = 0
WireCrypt = Enabled
ExtConnPoolSize = 10
ExtConnPoolLifeTime = 10
UseFileSystemCache = true
IpcName = xnet_fb4x_qa
RemotePipeName = wnet_fb4x_qa
# Recommended
DefaultDbCachePages = 10000
MaxUnflushedWrites = -1
MaxUnflushedWriteTime = -1
# number of seconds after which statement execution will be automatically cancelled
# by the engine
# can be very useful if some test will hang or become work extremely slow:
StatementTimeout = 7200
# Needed for encryption-related tests:
KeyHolderPlugin = fbSampleKeyHolder
```
**Firebird 5:**
currently all parameters from FB-4.x can be used, except 'RemotePipeName'
because support of WNET protocol was removed from FB-5.x.
It is recommended to assign value like 'xnet_fb5x_qa' to IpcName.
> [!NOTE]
>
> Proper values of some parameters strongly depends on ServerMode and amount of
> avaliable RAM.
>
> * DefaultDbCachePages::
> On Classic/SuperClassic it must not be greater than 4K in real systems. For tests
> 10K...20K is OK.
> On Super it can be increased so that size of page cache become 25%...33% of
> physical RAM.
> * TempCacheLimit::
> On Classic usually it must be limited because every connection uses its own memory area
> for sort purposes. Nowadays may to use values about 256M ... 512M.
> On SuperClassic and Super common memory area is used to sorts, so this parameter can have
> values of dozen Gb. As first approximation, it can be set up to 33% of total RAM.
2. Changes in OS environment variables:
it is recommended to create variable `FIREBIRD_TMP` that will point to the pre-created directory
on some fast drive (e.g. SSD or RAM). This drive must have at least 30 Gb free space.
Once this variable is defined, one may not specify parameter 'TempDirectories'.
3. Required changes for running encryption-related tests::
1. Change dir to `$FB_HOME/examples/prebuilt/plugins/` and make copy of following files into `$FB_HOME/plugins/` :
1. Configs:
```
fbSampleKeyHolder.conf
fbSampleDbCrypt.conf
```
2. Binaries on Windows:
```
fbSampleDbCrypt.dll
fbSampleKeyHolder.dll
```
3. Binaries on Linux:
```
libfbSampleDbCrypt.so
libfbSampleKeyHolder.so
```
> [!NOTE]
>
> These files missed in Firebird 3.x but one may to use such files from any
> recent FB 4.x snapshot.
> Config parameter KeyHolderPlugin currently is 'fbSampleKeyHolder'.
> This value must match to value of parameter 'ENCRYPTION_HOLDER' that is
> specified in the file `$(QA_ROOT)/files/test_config.ini` (it contains several settings that are common for many tests).
2. In `$FB_HOME/plugins/fbSampleKeyHolder.conf`:
```
Auto = true
KeyRed=111
KeyGreen = 119
```
3. In `$FB_HOME/plugins/fbSampleDbCrypt.conf`:
```
# Encure that Auto = false or just is commented out
```
4. Restart Firebird and check that all set correct. Example for Linux:
```
shell rm -f /var/tmp/tmp4test.fdb;
create database 'localhost:/var/tmp/tmp4test.fdb' user sysdba password 'masterkey';
```
**Following must PASS:**
```
set echo on;
set bail on;
alter database encrypt with "fbSampleDbCrypt" key Red;
shell sleep 2;
alter database decrypt;
shell sleep 2;
alter database encrypt with "fbSampleDbCrypt" key Green;
shell sleep 2;
alter database decrypt;
shell sleep 2;
set echo off;
set bail off;
```
**Following must FAIL with:**
```
-- Statement failed, SQLSTATE = 42000
-- unsuccessful metadata update
-- -ALTER DATABASE failed
-- -Missing correct crypt key
-- -Plugin fbSampleKeyHolder:
-- -Crypt key NOSUCH not set
set echo on;
alter database encrypt with "fbSampleDbCrypt" key NoSuch;
shell sleep 2;
show database;
quit;
```
> [!IMPORTANT]
> Ensure that EMPLOYEE database was not encrypted before with key/value that is unknown currently!
> Otherwise attempt to run ANY test will fail with:
> ```
> INTERNALERROR> firebird.driver.types.DatabaseError: Missing database encryption key for your attachment
> INTERNALERROR> -Plugin fbSampleKeyHolder:
> INTERNALERROR> -Crypt key <HERE_SOME_UNKNOWN_KEY> not set
> ```
4. Additional issues about folder `$(dir_sampleDb)` (`$FB_HOME/examples/empbuild/`) and its subdirectories.
1. There are many tests which supposes that this directory is avaliable for read/write access.
Test suite (firebird-qa plugin for pytest) will re-create subdirectory with name 'qa' under `$(dir_sample)` for
every such test, so be sure that you have not any significant data in this folder.
2. Firebird 4.x+ has ability to involve databases in replication schema. There are several tests which assumes that
such schema already was created (before pytest session) and there arte two databases in it (master and replica).
It was decided to use directory with name: `$(dir_sampleDb)/qa_replication/` for this purpoces. Two databases must
be created in it: `db_main.fdb` and `db_repl.fdb`, and one need to prepare them into replication beforehand.
Also, one need to prepare two directories in THIS folder which will serve as replication journal and archive.
All these actions are performed by batch scenarios which can be provided by IBSurgeon company by request.
4. Optional. Enable your OS to create dump files in case of FB crashes caused by tests::
1. Windows:
1. Parameter `BugcheckAbort` must always be set to 1, otherwise dumps will not be created.
2. Run regedit, navigate to key:
```
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\
```
Create sub-key there with name: 'firebird.exe' (without single quotes).
Add following parameters in the 'firebird.exe' key:
```
DumpCount, type = DWORD, value: not less than 5;
DumpFoler, type = REG_EXPAND_SZ, value = directory where you want dumps to be created;
DumpType, type = DWORD, value = 2
```
3. Following setting must present in the registry to disable any pop-up window when program crashes:
key: `HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Windows Error Reporting\`
parameter: `'DontShowUI', type = DWORD, value: 2`
4.2. Linux:
File `/etc/security/limits.conf` must have setting:
```
* soft core unlimited
```
File `/etc/sysctl.conf` must have parameter `kernel.core_pattern` that specifies
directory to store dumps and pattern for dumps name, e.g.:
```
kernel.core_pattern=/var/tmp/core.%e.%t.%p
```
5. Cautions.
1. Problems can occur on Windows if we launch two FB instances which uses the same major version ODS.
Currently this relates to FB-4.x and FB-5.x: each of them tries to create file 'fb13_user_mapping'
in `%programdata%\firebird`. This leads to conflict and attempt to connect to any DB using latter FB instance
issues "Error occurred during login, please check server firebird.log for details" and firebird.log will have:
"Database is probably already opened by another engine instance in another Windows session".
BE SURE THAT YOU DID NOT LAUNCH ANOTHER FIREBIRD INSTANCE THAT USES SAME ODS AS CURRENTLY TESTING.
2. Be sure that directory specified by FIREBIRD_TMP variable actually exists and is accessible for 'firebird' account.
3. Ensure that your `firebird-driver.conf` contains 'DEFAULT' section with `encoding_errors = ignore`.
Otherwise outcome of some tests can be unpredictable if your OS has non-ascii system console
6. Use pytest to run tests.
The plugin adds next options to pytest:
```
Firebird server:
--server=SERVER Server configuration name
--bin-dir=PATH Path to directory with Firebird utilities
--protocol={xnet,inet,inet4,wnet}
Network protocol used for database attachments
--runslow Run slow tests
--save-output Save test std[out|err] output to files
--skip-deselected={platform,version,any}
SKIP tests instead deselection
--extend-xml Extend XML JUnit report with additional information
--install-terminal Use our own terminal reporter
```
To run all tests (except slow ones) against local server use next command::
```
pytest --server local ./tests
```
> [!NOTE]
> If plugin fails to determine the directory with Firebird utilities (isql, gbak etc.),
> use `--bin-dir` option to specify it.

267
README.rst
View File

@ -1,267 +0,0 @@
===========
Firebird QA
===========
This package contains:
- pytest plugin that provides support for testing the Firebird engine. It uses new Python
driver for Firebird (firebird-driver).
- tests for Firebird engine (directory 'tests')
- files needed by tests (directories 'databases', 'files', 'backups' and 'configs')
Requirements: Python 3.8+, Firebird 3+
You should definitelly read the `QA suite documentation`_ !
Quickstart
----------
1. Clone the git repository
2. Install the plugin and required dependencies by running next command from repo. directory::
pip install -e .
3. Adjust Firebird server configuration.
3.0. ONLY FOR MANUAL runs::
Check content of $FB_HOME/databases.conf.
Ensure that RemoteAccess is allowed for security.db.
Also, it is recommended to set number of buffers not less than 256 for it:
security.db = $(dir_secDb)/security<suffix>.fdb
{
RemoteAccess = true
DefaultDbCachePages = 256
}
This must be done only if you want to run some tests manually.
Automated scenario for running tests will overwrite this file
and put there all needed data before every pytest session (using
$QA_ROOT/files/qa-databases.conf as prototype for that purpose).
3.1. $FB_HOME/firebird.conf::
Firebird 3::
# Required
BugcheckAbort = 1
ExternalFileAccess = Full
AuthServer = Srp, Win_Sspi, Legacy_Auth
UserManager = Srp, Legacy_UserManager
WireCrypt = Enabled
FileSystemCacheThreshold = 99999K
IpcName = xnet_fb3x_qa
RemotePipeName = wnet_fb3x_qa
# Recommended
DefaultDbCachePages = 10000
MaxUnflushedWrites = -1
MaxUnflushedWriteTime = -1
# Needed for encryption-related tests.
KeyHolderPlugin = fbSampleKeyHolder
Firebird 4::
# Required
BugcheckAbort = 1
ExternalFileAccess = Full
AuthServer = Srp, Win_Sspi, Legacy_Auth
UserManager = Srp, Legacy_UserManager
ReadConsistency = 0
WireCrypt = Enabled
ExtConnPoolSize = 10
ExtConnPoolLifeTime = 10
UseFileSystemCache = true
IpcName = xnet_fb4x_qa
RemotePipeName = wnet_fb4x_qa
# Recommended
DefaultDbCachePages = 10000
MaxUnflushedWrites = -1
MaxUnflushedWriteTime = -1
# number of seconds after which statement execution will be automatically cancelled by the engine
# can be very useful if some test will hang or become work extremely slow:
StatementTimeout = 7200
# Needed for encryption-related tests:
KeyHolderPlugin = fbSampleKeyHolder
Firebird 5::
currently all parameters from FB-4.x can be used, except 'RemotePipeName'
because support of WNET protocol was removed from FB-5.x.
It is recommended to assign value like 'xnet_fb5x_qa' to IpcName.
NOTES::
Proper values of some parameters strongly depends on ServerMode and amount of avaliable RAM.
* DefaultDbCachePages::
On Classic/SuperClassic it must not be greater than 4K in real systems. For tests 10K...20K is OK.
On Super it can be increased so that size of page cache become 25%...33% of physical RAM.
* TempCacheLimit::
On Classic usually it must be limited because every connection uses its own memory area
for sort purposes. Nowadays may to use values about 256M ... 512M.
On SuperClassic and Super common memory area is used to sorts, so this parameter can have
values of dozen Gb. As first approximation, it can be set up to 33% of total RAM.
3.2. Changes in OS environment variables::
it is recommended to create variable FIREBIRD_TMP that will point to the pre-created directory
on some fast drive (e.g. SSD or RAM). This drive must have at least 30 Gb free space.
Once this variable is defined, one may not specify parameter 'TempDirectories'.
3.3. Required changes for running encryption-related tests::
3.3.1. Change dir to $FB_HOME/examples/prebuilt/plugins/ and make copy of following files into $FB_HOME/plugins/ ::
3.3.1.1. Configs::
fbSampleKeyHolder.conf
fbSampleDbCrypt.conf
3.3.1.2. Binaries on Windows::
fbSampleDbCrypt.dll
fbSampleKeyHolder.dll
3.3.1.3. Binaries on Linux::
libfbSampleDbCrypt.so
libfbSampleKeyHolder.so
NOTES.
These files missed in Firebird 3.x but one may to use such files from any recent FB 4.x snapshot.
Config parameter KeyHolderPlugin currently is 'fbSampleKeyHolder'.
This value must match to value of parameter 'ENCRYPTION_HOLDER' that is specified in the file
$(QA_ROOT)/files/test_config.ini (it contains several settings that are common for many tests).
3.3.2. In $FB_HOME/plugins/fbSampleKeyHolder.conf::
Auto = true
KeyRed=111
KeyGreen = 119
3.3.3. In $FB_HOME/plugins/fbSampleDbCrypt.conf::
# Encure that Auto = false or just is commented out
3.3.3. Restart Firebird and check that all set correct. Example for Linux:
shell rm -f /var/tmp/tmp4test.fdb;
create database 'localhost:/var/tmp/tmp4test.fdb' user sysdba password 'masterkey';
-- Following must PASS:
set echo on;
set bail on;
alter database encrypt with "fbSampleDbCrypt" key Red;
shell sleep 2;
alter database decrypt;
shell sleep 2;
alter database encrypt with "fbSampleDbCrypt" key Green;
shell sleep 2;
alter database decrypt;
shell sleep 2;
set echo off;
set bail off;
-- Following must FAIL with:
-- Statement failed, SQLSTATE = 42000
-- unsuccessful metadata update
-- -ALTER DATABASE failed
-- -Missing correct crypt key
-- -Plugin fbSampleKeyHolder:
-- -Crypt key NOSUCH not set
set echo on;
alter database encrypt with "fbSampleDbCrypt" key NoSuch;
shell sleep 2;
show database;
quit;
3.3.4. IMPORTANT.
Ensure that EMPLOYEE database was not encrypted before with key/value that is unknown currently!
Otherwise attempt to run ANY test will fail with:
INTERNALERROR> firebird.driver.types.DatabaseError: Missing database encryption key for your attachment
INTERNALERROR> -Plugin fbSampleKeyHolder:
INTERNALERROR> -Crypt key <HERE_SOME_UNKNOWN_KEY> not set
3.4. Additional issues about folder $(dir_sampleDb) ( $FB_HOME/examples/empbuild/ ) and its subdirectories.
3.4.1. There are many tests which supposes that this directory is avaliable for read/write access.
Test suite (firebird-qa plugin for pytest) will re-create subdirectory with name 'qa' under $(dir_sample) for
every such test, so be sure that you have not any significant data in this folder.
3.4.2. Firebird 4.x+ has ability to involve databases in replication schema. There are several tests which assumes that
such schema already was created (before pytest session) and there arte two databases in it (master and replica).
It was decided to use directory with name: $(dir_sampleDb)/qa_replication/ for this purpoces. Two databases must
be created in it: db_main.fdb and db_repl.fdb, and one need to prepare them into replication beforehand.
Also, one need to prepare two directories in THIS folder which will serve as replication journal and archive.
All these actions are performed by batch scenarios which can be provided by IBSurgeon company by request.
4. Optional. Enable your OS to create dump files in case of FB crashes caused by tests::
4.1. Windows::
4.1.1. Parameter 'BugcheckAbort' must always be set to 1, otherwise dumps will not be created.
4.1.2. Run regedit, navigate to key::
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\
Create sub-key there with name: 'firebird.exe' (without single quotes).
Add following parameters in the 'firebird.exe' key::
DumpCount, type = DWORD, value:: not less than 5;
DumpFoler, type = REG_EXPAND_SZ, value = directory where you want dumps to be created;
DumpType, type = DWORD, value = 2
4.1.3. Following setting must present in the registry to disable any pop-up window when program crashes::
key": HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Windows Error Reporting\
parameter:: 'DontShowUI', type = DWORD, value:: 2
4.2. Linux::
File /etc/security/limits.conf must have setting::
* soft core unlimited
File /etc/sysctl.conf must have parameter 'kernel.core_pattern' that specifies directory to store dumps
and pattern for dumps name, e.g.::
kernel.core_pattern=/var/tmp/core.%e.%t.%p
5. Cautions.
5.1. Problems can occur on Windows if we launch two FB instances which uses the same major version ODS.
Currently this relates to FB-4.x and FB-5.x: each of them tries to create file 'fb13_user_mapping'
in %programdata%\firebird. This leads to conflict and attempt to connect to any DB using latter FB instance
issues "Error occurred during login, please check server firebird.log for details" and firebird.log will have:
"Database is probably already opened by another engine instance in another Windows session".
BE SURE THAT YOU DID NOT LAUNCH ANOTHER FIREBIRD INSTANCE THAT USES SAME ODS AS CURRENTLY TESTING.
5.2. Be sure that directory specified by FIREBIRD_TMP variable actually exists and is accessible for 'firebird' account.
5.3. Ensure that your firebird-driver.conf contains 'DEFAULT' section with 'encoding_errors = ignore'.
Otherwise outcome of some tests can be unpredictable if your OS has non-ascii system console
6. Use pytest to run tests.
The plugin adds next options to pytest::
Firebird server:
--server=SERVER Server configuration name
--bin-dir=PATH Path to directory with Firebird utilities
--protocol={xnet,inet,inet4,wnet}
Network protocol used for database attachments
--runslow Run slow tests
--save-output Save test std[out|err] output to files
--skip-deselected={platform,version,any}
SKIP tests instead deselection
--extend-xml Extend XML JUnit report with additional information
--install-terminal Use our own terminal reporter
To run all tests (except slow ones) against local server use next command::
pytest --server local ./tests
Note:
If plugin fails to determine the directory with Firebird utilities (isql, gbak etc.),
use `--bin-dir` option to specify it.
.. _QA suite documentation: https://firebird-qa.readthedocs.io

8
docs/changelog.txt
View File

@ -4,6 +4,12 @@ Changelog
.. currentmodule:: firebird.qa.plugin
Version 0.19.0
==============
* Switch from `setuptools` to `hatch`.
* Updated dependencies.
Version 0.18.0
==============
@ -19,7 +25,7 @@ Version 0.18.0
Version 0.17.3
==============
* Added --driver-config option to specify different filename for driver configuration.
* Added `--driver-config` option to specify different filename for driver configuration.
Version 0.17.2
==============

7
docs/conf.py
View File

@ -15,18 +15,19 @@
# sys.path.insert(0, os.path.abspath('.'))
import sphinx_bootstrap_theme
from firebird.qa.__about__ import __version__
# -- Project information -----------------------------------------------------
project = 'Firebird QA'
copyright = '2021, Pavel Cisar'
copyright = '2021-present, Pavel Císař'
author = 'Pavel Císař'
# The short X.Y version
version = '0.17.2'
version = __version__
# The full version, including alpha/beta/rc tags
release = '0.17.2'
release = __version__
# -- General configuration ---------------------------------------------------

205
pyproject.toml
View File

@ -1,3 +1,204 @@
[build-system]
requires = ["setuptools >= 58.0.0", "wheel"]
build-backend = "setuptools.build_meta"
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
name = "firebird-qa"
dynamic = ["version"]
description = "pytest plugin for Firebird QA"
readme = "PLUGIN-README.md"
requires-python = ">=3.8, <4"
license = { file = "LICENSE" }
authors = [
{ name = "Pavel Cisar", email = "pcisar@users.sourceforge.net"},
]
keywords = ["Firebird", "Logging", "Trace", "Configuration", "Signals", "Protobuf", "Hooks",
"Collections"]
classifiers = [
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Operating System :: POSIX :: Linux",
"Operating System :: Microsoft :: Windows",
"Operating System :: MacOS",
"Topic :: Database :: Database Engines/Servers",
"Topic :: Software Development :: Testing",
"Framework :: Pytest",
]
dependencies = [
"firebird-base>=1.7.1",
"firebird-driver>=1.10.1",
"pytest>=8.0.0",
"psutil>=5.9.8",
]
[project.urls]
Home = "https://github.com/FirebirdSQL/firebird-qa"
Documentation = "https://firebird-qa.rtfd.io"
Issues = "https://github.com/FirebirdSQL/firebird-qa/issues"
Funding = "https://github.com/sponsors/pcisar"
Source = "https://github.com/FirebirdSQL/firebird-qa"
[project.scripts]
fbt-conv = "firebird.qa.fbtconv:main"
[project.entry-points.pytest11]
firebird = "firebird.qa.plugin"
[tool.hatch.version]
path = "src/firebird/qa/__about__.py"
[tool.hatch.build.targets.sdist]
include = ["src"]
[tool.hatch.build.targets.wheel]
packages = ["src/firebird"]
[tool.hatch.envs.default]
dependencies = [
]
[tool.hatch.envs.test]
dependencies = [
"coverage[toml]>=6.5",
"pytest",
]
[tool.hatch.envs.test.scripts]
test = "pytest {args:tests}"
test-cov = "coverage run -m pytest {args:tests}"
cov-report = [
"- coverage combine",
"coverage report",
]
cov = [
"test-cov",
"cov-report",
]
version = "python --version"
[[tool.hatch.envs.test.matrix]]
python = ["3.8", "3.9", "3.10", "3.11", "3.12"]
[tool.hatch.envs.doc]
detached = false
platforms = ["linux"]
dependencies = [
"Sphinx>=7.2.6",
"sphinx-bootstrap-theme>=0.8.1",
"sphinx-autodoc-typehints>=1.24.0",
"doc2dash>=3.0.0"
]
[tool.hatch.envs.doc.scripts]
build = "cd docs ; make html"
docset = [
"cd docs ; doc2dash -u https://firebird-qa.readthedocs.io/en/latest/ -f -i ./_static/fb-favicon.png -n firebird-qa ./_build/html/",
"cd docs; VERSION=`hatch version` ; tar --exclude='.DS_Store' -cvzf ../dist/firebird-qa-$VERSION-docset.tgz firebird-qa.docset",
]
[tool.hatch.envs.lint]
detached = true
dependencies = [
"black>=23.1.0",
"mypy>=1.0.0",
"ruff>=0.0.243",
]
[tool.hatch.envs.lint.scripts]
typing = "mypy --install-types --non-interactive {args:src/firebird/qa tests}"
style = [
"ruff {args:.}",
"black --check --diff {args:.}",
]
fmt = [
"black {args:.}",
"ruff --fix {args:.}",
"style",
]
all = [
"style",
"typing",
]
[tool.black]
target-version = ["py38"]
line-length = 120
skip-string-normalization = true
[tool.ruff]
target-version = "py38"
line-length = 120
select = [
"A",
"ARG",
"B",
"C",
"DTZ",
"E",
"EM",
"F",
"FBT",
"I",
"ICN",
"ISC",
"N",
"PLC",
"PLE",
"PLR",
"PLW",
"Q",
"RUF",
"S",
"T",
"TID",
"UP",
"W",
"YTT",
]
ignore = [
# Allow non-abstract empty methods in abstract base classes
"B027",
# Allow boolean positional values in function calls, like `dict.get(... True)`
"FBT003",
# Ignore checks for possible passwords
"S105", "S106", "S107",
# Ignore complexity
"C901", "PLR0911", "PLR0912", "PLR0913", "PLR0915",
]
unfixable = [
# Don't touch unused imports
"F401",
]
[tool.ruff.isort]
known-first-party = ["firebird.qa"]
[tool.ruff.flake8-tidy-imports]
ban-relative-imports = "all"
[tool.ruff.per-file-ignores]
# Tests can use magic values, assertions, and relative imports
"tests/**/*" = ["PLR2004", "S101", "TID252"]
[tool.coverage.run]
source_pkgs = ["firebird.qa", "tests"]
branch = true
parallel = true
omit = [
"src/firebird/qa/__about__.py",
]
[tool.coverage.paths]
firebird_qa = ["src/python", "*/python/src/firebird/qa"]
tests = ["tests", "*/python/tests"]
[tool.coverage.report]
exclude_lines = [
"no cov",
"if __name__ == .__main__.:",
"if TYPE_CHECKING:",
]

61
setup.cfg
View File

@ -1,61 +0,0 @@
[build_sphinx]
source-dir=docs
all-files=True
[metadata]
name = firebird-qa
version = 0.19.0
description = pytest plugin for Firebird QA
long_description = file: README.rst
long_description_content_type = text/x-rst; charset=UTF-8
author = Pavel Císař
author_email = pcisar@users.sourceforge.net
license = MIT
license_files = LICENSE
url = https://github.com/FirebirdSQL/fbtest
keywords = Firebird RDBMS QA tools
project_urls =
Documentation = https://firebird-qa.rtfd.io
Bug Reports = https://github.com/FirebirdSQL/firebird-qa/issues
Funding = https://www.firebirdsql.org/en/donate/
Source = https://github.com/FirebirdSQL/firebird-qa
classifiers =
Development Status :: 5 - Production/Stable
Intended Audience :: Developers
License :: OSI Approved :: MIT License
Programming Language :: Python :: 3
Programming Language :: Python :: 3.8
Operating System :: POSIX :: Linux
Operating System :: Microsoft :: Windows
Operating System :: MacOS
Topic :: Software Development :: Testing
Topic :: Database
Framework :: Pytest
[options]
zip_safe = True
python_requires = >=3.8, <4
install_requires =
firebird-base>=1.5.0
firebird-driver>=1.8.0
pytest>=7.0.0
psutil>=5.9.1
packages = find_namespace:
[options.packages.find]
include = firebird.*
[options.entry_points]
pytest11 =
firebird = firebird.qa.plugin
console_scripts =
fbt-conv = firebird.qa.fbtconv:main
[bdist_wheel]
# This flag says to generate wheels that support both Python 2 and Python
# 3. If your code will not run unchanged on both Python 2 and 3, you will
# need to generate separate wheels for each Python version that you
# support.
universal=0

10
setup.py
View File

@ -1,10 +0,0 @@
#!/usr/bin/env python
#coding:utf-8
# This file is only a shim to allow editable installs. It's not necessary to build
# and install the package via pip (see pyproject.toml and setup.cfg).
import setuptools
if __name__ == "__main__":
setuptools.setup()

4
src/firebird/qa/__about__.py Normal file
View File

@ -0,0 +1,4 @@
# SPDX-FileCopyrightText: 2021-present The Firebird Projects <www.firebirdsql.org>
#
# SPDX-License-Identifier: MIT
__version__ = "0.19.0"

4
firebird/qa/__init__.py → src/firebird/qa/__init__.py
View File

@ -1,4 +1,6 @@
#coding:utf-8
# SPDX-FileCopyrightText: 2021-present The Firebird Projects <www.firebirdsql.org>
#
# SPDX-License-Identifier: MIT
#
# PROGRAM/MODULE: firebird-qa
# FILE: firebird/qa/__init__.py

4
firebird/qa/fbtconv.py → src/firebird/qa/fbtconv.py
View File

@ -1,4 +1,6 @@
#coding:utf-8
# SPDX-FileCopyrightText: 2021-present The Firebird Projects <www.firebirdsql.org>
#
# SPDX-License-Identifier: MIT
#
# PROGRAM/MODULE: firebird-qa
# FILE: firebird/qa/fbtconv.py

4
firebird/qa/plugin.py → src/firebird/qa/plugin.py
View File

@ -1,4 +1,6 @@
#coding:utf-8
# SPDX-FileCopyrightText: 2021-present The Firebird Projects <www.firebirdsql.org>
#
# SPDX-License-Identifier: MIT
#
# PROGRAM/MODULE: firebird-qa
# FILE: firebird/qa/plugin.py