mirror of
https://github.com/FirebirdSQL/firebird-qa.git
synced 2025-01-23 05:53:06 +01:00
Added/Updated tests\bugs\core_0733_test.py: Fixed wrong usage of driver_config.register_database() instance, added check for properly selected protocol. See other notes in the test source.
This commit is contained in:
pavel-zotov
parent
876164c853
commit
0aa8320919
@ -5,51 +5,55 @@ ID: issue-1108
|
||||
ISSUE: 1108
|
||||
TITLE: Compress Data over the Network
|
||||
DESCRIPTION:
|
||||
Test creates two tables with similar DDL (single varchar column) and fills data into them.
|
||||
First table ('t_common_text') gets textual data which can be compressed in extremely high degree.
|
||||
Second table ('t_binary_data') will store GUID-data which is obviously incompressable.
|
||||
Then we perform <N_MEASURES> runs with selecting data from each of these tables, and we store
|
||||
result of *deltas* of CPU user time that was 'captured' before and after cursor fetch all rows.
|
||||
Test creates two tables with similar DDL (single varchar column) and fills data into them.
|
||||
First table ('t_common_text') gets textual data which can be compressed in extremely high degree.
|
||||
Second table ('t_binary_data') will store GUID-data which is obviously incompressable.
|
||||
Then we perform <N_MEASURES> runs with selecting data from each of these tables, and we store
|
||||
result of *deltas* of CPU user time that was 'captured' before and after cursor fetch all rows.
|
||||
|
||||
Within each measurement we gather info TWO times, by changing WireCompression = true vs false
|
||||
(this is client-side parameter and it must be defined in DPB; see 'db_cfg_object' variable).
|
||||
Within each measurement we gather info TWO times, by changing WireCompression = true vs false
|
||||
(this is client-side parameter and it must be defined in DPB; see 'db_cfg_object' variable).
|
||||
|
||||
Results of <N_MEASURES> for each value of WireCompression and each source (t_common_text vs t_binary_data)
|
||||
are accumulated in the lists, and MEDIAN (of that CPU time values) is evaluated after all serie completed.
|
||||
Results of <N_MEASURES> for each value of WireCompression and each source (t_common_text vs t_binary_data)
|
||||
are accumulated in the lists, and MEDIAN (of that CPU time values) is evaluated after all serie completed.
|
||||
|
||||
Following ratios are compared in this test with apropriate 'thresholds':
|
||||
1) how CPU UserTime depends on WireCompression if we send 'IDEALLY COMPRESSABLE' textual data over network.
|
||||
If WireCompression is ON (and actually works) then CPU time must be GREATER then if WireCompression = OFF.
|
||||
Minimal threshold for that case is tuned by 'MIN_CPU_RATIO_TXT_WCOMPR_ON_OFF' setting;
|
||||
2) same as "1", but when we send 'ABSOLUTELY IMCOMPRESSABLE' binary data over network.
|
||||
Minimal threshold for that case is tuned by 'MIN_CPU_RATIO_BIN_WCOMPR_ON_OFF' setting;
|
||||
3) how CPU userTime depends on NATURE of sending data ('ideally compressable' vs 'absolutely imcompressable')
|
||||
if WireCompression is OFF. This comparison is not mandatory for this test purpose, but it can be useful
|
||||
for estimation of how record-level compression works.
|
||||
Ideally compresable text occupies much less data pages this CPU usertime for processing must be LESS then
|
||||
for imcompressible binary data.
|
||||
Maximal ratio between them must be more than 1, see setting 'MAX_CPU_RATIO_TXT2BIN_WCOMPR_OFF'.
|
||||
Following ratios are compared in this test with apropriate 'thresholds':
|
||||
1) how CPU UserTime depends on WireCompression if we send 'IDEALLY COMPRESSABLE' textual data over network.
|
||||
If WireCompression is ON (and actually works) then CPU time must be GREATER then if WireCompression = OFF.
|
||||
Minimal threshold for that case is tuned by 'MIN_CPU_RATIO_TXT_WCOMPR_ON_OFF' setting;
|
||||
2) same as "1", but when we send 'ABSOLUTELY IMCOMPRESSABLE' binary data over network.
|
||||
Minimal threshold for that case is tuned by 'MIN_CPU_RATIO_BIN_WCOMPR_ON_OFF' setting;
|
||||
3) how CPU userTime depends on NATURE of sending data ('ideally compressable' vs 'absolutely imcompressable')
|
||||
if WireCompression is OFF. This comparison is not mandatory for this test purpose, but it can be useful
|
||||
for estimation of how record-level compression works.
|
||||
Ideally compresable text occupies much less data pages this CPU usertime for processing must be LESS then
|
||||
for imcompressible binary data.
|
||||
Maximal ratio between them must be more than 1, see setting 'MAX_CPU_RATIO_TXT2BIN_WCOMPR_OFF'.
|
||||
|
||||
We have to compare RATIOS between these CPU time medians with some statistics which was collected beforhand.
|
||||
Ten runs was done for Windows and Linux, values of thresholds see in the source below.
|
||||
We have to compare RATIOS between these CPU time medians with some statistics which was collected beforhand.
|
||||
Ten runs was done for Windows and Linux, values of thresholds see in the source below.
|
||||
|
||||
See also tests for:
|
||||
CORE-5536 - checks that field mon$wire_compressed actually exists in MON$ATTACHMENTS table;
|
||||
CORE-5913 - checks that built-in rdb$get_context('SYSTEM','WIRE_ENCRYPTED') is avaliable;
|
||||
See also tests for:
|
||||
CORE-5536 - checks that field mon$wire_compressed actually exists in MON$ATTACHMENTS table;
|
||||
CORE-5913 - checks that built-in rdb$get_context('SYSTEM','WIRE_ENCRYPTED') is avaliable;
|
||||
NOTES:
|
||||
[09.11.2021] pcisar
|
||||
This test was fragile from start, usualy lefts behind resources and requires
|
||||
temporary changes to firebird.conf on run-time. It's questionable whether
|
||||
wire-compression should be tested at all.
|
||||
[09.11.2021] pcisar
|
||||
This test was fragile from start, usualy lefts behind resources and requires
|
||||
temporary changes to firebird.conf on run-time. It's questionable whether
|
||||
wire-compression should be tested at all.
|
||||
|
||||
[04.08.2022] pzotov
|
||||
Fully re-implemented (see description). No more datediff() for time measurement, CPU UserTime is analyzed.
|
||||
[04.08.2022] pzotov
|
||||
Fully re-implemented (see description). No more datediff() for time measurement, CPU UserTime is analyzed.
|
||||
Checked on 5.0.0.591, 4.0.1.2692, 3.0.8.33535 - both on Windows and Linux.
|
||||
|
||||
CAUTION.
|
||||
DO NOT set DATA_WIDTH less than 1'500 and N_ROWS less then 10'000
|
||||
otherwise ratios can be unreal because of CPU UserTime = 0.
|
||||
[03.03.2023] pzotov
|
||||
Fixed wrong usage of driver_config.register_database() instance, added check for properly selected protocol (only INET must be used).
|
||||
Added columns to GTT in order to see in the trace used values of compression_suitability and wire_compression.
|
||||
Checked on Windows: 5.0.0.967 SS/CS, 4.0.3.2904 SS/CS, 3.0.11.33665 SS/CS
|
||||
|
||||
Checked on 5.0.0.591, 4.0.1.2692, 3.0.8.33535 - both on Windows and Linux.
|
||||
CAUTION.
|
||||
DO NOT set DATA_WIDTH less than 1'500 and N_ROWS less then 10'000
|
||||
otherwise ratios can be unreal because of CPU UserTime = 0.
|
||||
|
||||
JIRA: CORE-733
|
||||
FBTEST: bugs.core_0733
|
||||
@ -60,8 +64,7 @@ from collections import defaultdict
|
||||
|
||||
import pytest
|
||||
from firebird.qa import *
|
||||
from firebird.driver import driver_config, connect
|
||||
|
||||
from firebird.driver import driver_config, connect, NetProtocol
|
||||
#--------------------------------------------------------------------
|
||||
def median(lst):
|
||||
n = len(lst)
|
||||
@ -145,30 +148,41 @@ def test_1(act: Action, capsys):
|
||||
MAX_CPU_RATIO_TXT2BIN_WCOMPR_OFF = 1.1 if platform.system() == 'Linux' else 0.95
|
||||
|
||||
# Register Firebird server (D:\FB\probes\fid-set-dpb-probe-05x.py)
|
||||
srv_config_key_value_text = \
|
||||
f"""
|
||||
[test_srv_core_0733]
|
||||
protocol = inet
|
||||
"""
|
||||
driver_config.register_server(name = 'test_srv_core_0733', config = srv_config_key_value_text)
|
||||
db_cfg_object = driver_config.register_database(name = 'test_db_core_0733')
|
||||
db_cfg_object.database.value = str(act.db.db_path)
|
||||
# 4debug only, check in trace: db_cfg_object.charset.value = 'win1257'
|
||||
srv_cfg = driver_config.register_server(name = 'test_srv_core_0733', config = '')
|
||||
|
||||
#srv_config_key_value_text = \
|
||||
#f"""
|
||||
# [test_srv_core_0733]
|
||||
# protocol = inet
|
||||
#"""
|
||||
#driver_config.register_server(name = 'test_srv_core_0733', config = srv_config_key_value_text)
|
||||
#db_cfg_object = driver_config.register_database(name = 'test_db_core_0733')
|
||||
#db_cfg_object.database.value = str(act.db.db_path)
|
||||
|
||||
benchmark_data = defaultdict(list)
|
||||
for i in range(0, N_MEASURES):
|
||||
|
||||
for w_compr in ('true', 'false'):
|
||||
for w_compr in (True, False):
|
||||
|
||||
db_cfg_object.config.value = f"""
|
||||
WireCompression = {w_compr}
|
||||
WireCrypt = Disabled
|
||||
"""
|
||||
db_cfg_name = f'test_db_core_0733__wcompr_{w_compr}'
|
||||
db_cfg_object = driver_config.register_database(name = db_cfg_name)
|
||||
db_cfg_object.server.value = srv_cfg.name
|
||||
# db_cfg_object.protocol.value = None / XNET / WNET -- these can not be used in this test!
|
||||
db_cfg_object.protocol.value = NetProtocol.INET
|
||||
db_cfg_object.database.value = str(act.db.db_path)
|
||||
db_cfg_object.config.value = f"""
|
||||
WireCrypt = Disabled
|
||||
WireCompression = {w_compr}
|
||||
"""
|
||||
for i in range(0, N_MEASURES):
|
||||
with connect(db_cfg_name, user = act.db.user, password = act.db.password) as con:
|
||||
prot_name = db_cfg_object.protocol.value.name if db_cfg_object.protocol.value else 'Embedded'
|
||||
assert w_compr == con.info.is_compressed(), f'Protocol: {prot_name} - can not be used to measure wire_compression'
|
||||
|
||||
with connect('test_db_core_0733') as con:
|
||||
with con.cursor() as cur:
|
||||
cur.execute('select mon$server_pid as p from mon$attachments where mon$attachment_id = current_connection')
|
||||
fb_pid = int(cur.fetchone()[0])
|
||||
cur.execute('select mon$server_pid, mon$remote_protocol as p from mon$attachments where mon$attachment_id = current_connection')
|
||||
for r in cur:
|
||||
fb_pid, connection_protocol = r
|
||||
assert connection_protocol.startswith('TCP'), f'Invalid connection protocol: {connection_protocol}'
|
||||
|
||||
for data_source in ('t_common_text', 't_binary_data'):
|
||||
cur.execute('select s from ' + data_source)
|
||||
@ -179,14 +193,14 @@ def test_1(act: Action, capsys):
|
||||
pass
|
||||
fb_info_b = psutil.Process(fb_pid).cpu_times()
|
||||
|
||||
k = ('WireCompression=' + w_compr, 'DataSource='+data_source)
|
||||
k = ('WireCompression=%d' % (1 if w_compr else 0), 'DataSource='+data_source)
|
||||
benchmark_data[k] += fb_info_b.user - fb_info_a.user,
|
||||
|
||||
|
||||
compressed_txt = [ v for k,v in benchmark_data.items() if k[0] == 'WireCompression=true' and k[1] == 'DataSource=t_common_text'][0]
|
||||
compressed_bin = [ v for k,v in benchmark_data.items() if k[0] == 'WireCompression=true' and k[1] == 'DataSource=t_binary_data'][0]
|
||||
non_comprs_txt = [ v for k,v in benchmark_data.items() if k[0] == 'WireCompression=false' and k[1] == 'DataSource=t_common_text'][0]
|
||||
non_comprs_bin = [ v for k,v in benchmark_data.items() if k[0] == 'WireCompression=false' and k[1] == 'DataSource=t_binary_data'][0]
|
||||
compressed_txt = [ v for k,v in benchmark_data.items() if k[0] == 'WireCompression=1' and k[1] == 'DataSource=t_common_text'][0]
|
||||
compressed_bin = [ v for k,v in benchmark_data.items() if k[0] == 'WireCompression=1' and k[1] == 'DataSource=t_binary_data'][0]
|
||||
non_comprs_txt = [ v for k,v in benchmark_data.items() if k[0] == 'WireCompression=0' and k[1] == 'DataSource=t_common_text'][0]
|
||||
non_comprs_bin = [ v for k,v in benchmark_data.items() if k[0] == 'WireCompression=0' and k[1] == 'DataSource=t_binary_data'][0]
|
||||
|
||||
cpu_txt_wcompr_ON_vs_OFF = median(compressed_txt) / max(0.00001, median(non_comprs_txt))
|
||||
cpu_bin_wcompr_ON_vs_OFF = median(compressed_bin) / max(0.00001, median(non_comprs_bin))
|
||||
|
||||
Loading…
Reference in New Issue
Block a user