Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: apache/tsfile
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: develop
Choose a base ref
...
head repository: apache/tsfile
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: feat/parallel-toggles
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 8 commits
  • 7 files changed
  • 1 contributor

Commits on Nov 12, 2026

  1. feat(cpp/python): expose the parallel read/write switches

    Add tsfile_set_parallel_read_enabled, tsfile_set_parallel_write_enabled
    and tsfile_set_thread_count to the C API, and the matching
    set_parallel_read_enabled / set_parallel_write_enabled / set_thread_count
    to the Python binding.
    
    The switches already existed as C++ globals; only the bindings were
    missing. They matter for benchmarking and for single-table workloads,
    where running the read path with the pool disabled measured ~20% faster
    than the default because the worker pool is mostly idle.
    ColinLeeo committed Nov 12, 2026
    Configuration menu
    Copy the full SHA
    f5ea638 View commit details
    Browse the repository at this point in the history
  2. test(cpp/python): cover the parallel read/write switches

    C API: verify the setters take effect on the shared config, that thread
    counts outside [1, 64] are rejected without touching the configured
    value, and that flipping the parallel read switch over the same file
    returns identical rows.
    
    Python: exercise the new bindings, including the InvalidArgumentError
    raised for an out-of-range thread count, and compare reads with the
    parallel path on and off.
    ColinLeeo committed Nov 12, 2026
    Configuration menu
    Copy the full SHA
    d3c96f1 View commit details
    Browse the repository at this point in the history
  3. feat(cpp/python): make the parallel switches observable and init-safe

    Follow-ups from reviewing the new switches:
    
    * Setting a parallel flag before the first reader/writer was silently
      undone: init_config_value() assigned both flags true during the lazy
      initialization performed by tsfile_reader_new/write_file_new.  Give
      them in-class defaults (as thread_count_ already has) and stop
      assigning them at init time.  Regression test sets both to 0, creates
      a file handle, and checks they stay 0.
    * set_thread_count() deleted the live pool before allocating its
      replacement, so a failed thread creation left g_thread_pool_ dangling
      and let the exception escape the extern "C" boundary.  Build the new
      pool first, swap it in, then delete the old one; map failures to
      RET_OOM and keep the previous pool.  Skip the join/respawn when the
      count is unchanged.
    * Add tsfile_get_parallel_read_enabled/..._write_enabled/
      tsfile_get_thread_count and the matching Python getters, and surface
      all three through get_tsfile_config()/set_tsfile_config().  The
      setter docs now state that the flags are read per operation and that
      no reader, writer or result set may be active when resizing the pool.
    * Update cpp/README.md and python/README.md, which documented
      non-existent entry points (storage::set_write_thread_count) and
      described auto-disable on single-core machines.
    ColinLeeo committed Nov 12, 2026
    Configuration menu
    Copy the full SHA
    6d4aebe View commit details
    Browse the repository at this point in the history
  4. fix(cpp/python): harden the thread-count setter and its Python check

    Catch (...) when building the replacement pool so no exception can cross
    the C API boundary, and accept integer-like values (numpy integers,
    IntEnum) in the Python config setter while still rejecting bool.
    ColinLeeo committed Nov 12, 2026
    Configuration menu
    Copy the full SHA
    fd0fda7 View commit details
    Browse the repository at this point in the history
  5. fix(python): reject huge thread counts with InvalidArgumentError

    set_thread_count() took int32_t, so Python-int-to-int32 conversion raised
    OverflowError before the library could return RET_INVALID_ARG. Accept an
    object, validate it (integer-like, excluding bool) and the [1, 64] range,
    then convert. Also declare ConfigValue's C++ bool fields as libcpp bool
    in the pxd instead of bint, so Cython sees the real field type.
    ColinLeeo committed Nov 12, 2026
    Configuration menu
    Copy the full SHA
    d00c6f1 View commit details
    Browse the repository at this point in the history
  6. test(cpp): give reader-test files a per-process unique name

    Parallel CTest processes can seed the name generator in the same clock
    tick; one then removes the file another is opening (E_ALREADY_EXIST at
    the fixture's open, observed as SEGFAULT / test failure on CI). Append
    pid and a per-process counter, as the writer tests already do.
    ColinLeeo committed Nov 12, 2026
    Configuration menu
    Copy the full SHA
    eb8d1bf View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2026

  1. test(cpp): give clock-seeded test files unique names

    Several fixtures build their .tsfile name from a clock count alone.  CTest
    runs cases in parallel processes, so two can read the same tick and then
    one removes the file the other is opening or reopening
    (E_ALREADY_EXIST / E_FILE_OPEN_ERR, seen as SEGFAULT on macOS).
    
    Append the process id and a per-process counter, the pattern the writer
    tests already use.  Covers table_read_failure, table_field_filter,
    chunk_reader_resource, aligned_chunk_reader_bulk,
    aligned_field_filter_regression, masked_bulk_copy and
    timeseries_schema_codec.
    ColinLeeo committed Nov 13, 2026
    Configuration menu
    Copy the full SHA
    efc71e2 View commit details
    Browse the repository at this point in the history
  2. Merge remote-tracking branch 'origin/test/unique-test-filenames' into…

    … feat/parallel-toggles
    ColinLeeo committed Nov 13, 2026
    Configuration menu
    Copy the full SHA
    dd84ca0 View commit details
    Browse the repository at this point in the history
Loading