Repository navigation
Comparing changes
Open a pull request
base repository: apache/tsfile
base: develop
head repository: apache/tsfile
compare: feat/parallel-toggles
- 8 commits
- 7 files changed
- 1 contributor
Commits on Nov 12, 2026
-
feat(cpp/python): expose the parallel read/write switches
Add tsfile_set_parallel_read_enabled, tsfile_set_parallel_write_enabled and tsfile_set_thread_count to the C API, and the matching set_parallel_read_enabled / set_parallel_write_enabled / set_thread_count to the Python binding. The switches already existed as C++ globals; only the bindings were missing. They matter for benchmarking and for single-table workloads, where running the read path with the pool disabled measured ~20% faster than the default because the worker pool is mostly idle.
Configuration menu - View commit details
-
Copy full SHA for f5ea638 - Browse repository at this point
Copy the full SHA f5ea638View commit details -
test(cpp/python): cover the parallel read/write switches
C API: verify the setters take effect on the shared config, that thread counts outside [1, 64] are rejected without touching the configured value, and that flipping the parallel read switch over the same file returns identical rows. Python: exercise the new bindings, including the InvalidArgumentError raised for an out-of-range thread count, and compare reads with the parallel path on and off.
Configuration menu - View commit details
-
Copy full SHA for d3c96f1 - Browse repository at this point
Copy the full SHA d3c96f1View commit details -
feat(cpp/python): make the parallel switches observable and init-safe
Follow-ups from reviewing the new switches: * Setting a parallel flag before the first reader/writer was silently undone: init_config_value() assigned both flags true during the lazy initialization performed by tsfile_reader_new/write_file_new. Give them in-class defaults (as thread_count_ already has) and stop assigning them at init time. Regression test sets both to 0, creates a file handle, and checks they stay 0. * set_thread_count() deleted the live pool before allocating its replacement, so a failed thread creation left g_thread_pool_ dangling and let the exception escape the extern "C" boundary. Build the new pool first, swap it in, then delete the old one; map failures to RET_OOM and keep the previous pool. Skip the join/respawn when the count is unchanged. * Add tsfile_get_parallel_read_enabled/..._write_enabled/ tsfile_get_thread_count and the matching Python getters, and surface all three through get_tsfile_config()/set_tsfile_config(). The setter docs now state that the flags are read per operation and that no reader, writer or result set may be active when resizing the pool. * Update cpp/README.md and python/README.md, which documented non-existent entry points (storage::set_write_thread_count) and described auto-disable on single-core machines.
Configuration menu - View commit details
-
Copy full SHA for 6d4aebe - Browse repository at this point
Copy the full SHA 6d4aebeView commit details -
fix(cpp/python): harden the thread-count setter and its Python check
Catch (...) when building the replacement pool so no exception can cross the C API boundary, and accept integer-like values (numpy integers, IntEnum) in the Python config setter while still rejecting bool.
Configuration menu - View commit details
-
Copy full SHA for fd0fda7 - Browse repository at this point
Copy the full SHA fd0fda7View commit details -
fix(python): reject huge thread counts with InvalidArgumentError
set_thread_count() took int32_t, so Python-int-to-int32 conversion raised OverflowError before the library could return RET_INVALID_ARG. Accept an object, validate it (integer-like, excluding bool) and the [1, 64] range, then convert. Also declare ConfigValue's C++ bool fields as libcpp bool in the pxd instead of bint, so Cython sees the real field type.
Configuration menu - View commit details
-
Copy full SHA for d00c6f1 - Browse repository at this point
Copy the full SHA d00c6f1View commit details -
test(cpp): give reader-test files a per-process unique name
Parallel CTest processes can seed the name generator in the same clock tick; one then removes the file another is opening (E_ALREADY_EXIST at the fixture's open, observed as SEGFAULT / test failure on CI). Append pid and a per-process counter, as the writer tests already do.
Configuration menu - View commit details
-
Copy full SHA for eb8d1bf - Browse repository at this point
Copy the full SHA eb8d1bfView commit details
Commits on Nov 13, 2026
-
test(cpp): give clock-seeded test files unique names
Several fixtures build their .tsfile name from a clock count alone. CTest runs cases in parallel processes, so two can read the same tick and then one removes the file the other is opening or reopening (E_ALREADY_EXIST / E_FILE_OPEN_ERR, seen as SEGFAULT on macOS). Append the process id and a per-process counter, the pattern the writer tests already use. Covers table_read_failure, table_field_filter, chunk_reader_resource, aligned_chunk_reader_bulk, aligned_field_filter_regression, masked_bulk_copy and timeseries_schema_codec.
Configuration menu - View commit details
-
Copy full SHA for efc71e2 - Browse repository at this point
Copy the full SHA efc71e2View commit details -
Merge remote-tracking branch 'origin/test/unique-test-filenames' into…
… feat/parallel-toggles
Configuration menu - View commit details
-
Copy full SHA for dd84ca0 - Browse repository at this point
Copy the full SHA dd84ca0View commit details
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff develop...feat/parallel-toggles