head 1.12; access; symbols pkgsrc-2026Q1:1.12.0.2 pkgsrc-2026Q1-base:1.12 pkgsrc-2025Q4:1.10.0.6 pkgsrc-2025Q4-base:1.10 pkgsrc-2025Q3:1.10.0.4 pkgsrc-2025Q3-base:1.10 pkgsrc-2025Q2:1.10.0.2 pkgsrc-2025Q2-base:1.10 pkgsrc-2025Q1:1.8.0.2 pkgsrc-2025Q1-base:1.8 pkgsrc-2024Q4:1.6.0.2 pkgsrc-2024Q4-base:1.6 pkgsrc-2024Q3:1.5.0.2 pkgsrc-2024Q3-base:1.5 pkgsrc-2024Q2:1.3.0.2 pkgsrc-2024Q2-base:1.3 pkgsrc-2024Q1:1.2.0.4 pkgsrc-2024Q1-base:1.2 pkgsrc-2023Q4:1.2.0.2 pkgsrc-2023Q4-base:1.2; locks; strict; comment @# @; 1.12 date 2026.02.01.08.40.04; author ryoon; state Exp; branches; next 1.11; commitid 5ScBnT2VfAlPpEsG; 1.11 date 2026.01.07.08.46.08; author wiz; state Exp; branches; next 1.10; commitid 1wQ3ICD8eebefrpG; 1.10 date 2025.06.15.13.30.19; author ryoon; state Exp; branches; next 1.9; commitid E1mRRDW58h6LrZYF; 1.9 date 2025.05.20.16.03.33; author ryoon; state Exp; branches; next 1.8; commitid TsNDVwsdNVr98FVF; 1.8 date 2025.02.26.15.42.51; author ryoon; state Exp; branches; next 1.7; commitid gt6SQyNMTMArGZKF; 1.7 date 2025.01.01.10.22.38; author ryoon; state Exp; branches; next 1.6; commitid qAnvl5v2cIccILDF; 1.6 date 2024.10.31.14.39.57; author ryoon; state Exp; branches; next 1.5; commitid EU9o47czOiXZ7PvF; 1.5 date 2024.09.17.06.10.43; author ryoon; state Exp; branches; next 1.4; commitid BQCe8PhZ85X1J7qF; 1.4 date 2024.08.25.06.18.28; author wiz; state Exp; branches; next 1.3; commitid BcrSL94SO85mvanF; 1.3 date 2024.06.05.18.22.39; author ryoon; state Exp; branches; next 1.2; commitid qK5HuBbXnOZn7PcF; 1.2 date 2023.12.01.15.29.29; author ryoon; state Exp; branches; next 1.1; commitid kmFz2wA7IYpFaMOE; 1.1 date 2023.12.01.10.45.58; author ryoon; state Exp; branches; next ; commitid 6JYUcoE7qpoqBKOE; desc @@ 1.12 log @databases/duckdb,py-duckdb: Update to 1.4.4 * Use separated tarballs for duckdb and py-duckdb. The tarball of duckdb does not contain py-duckdb. Changelog: 1.4.4: What's Changed Cast Fix: Correctly handle negative exponent with a number with a decimal in VARCHAR -> INTEGER cast by @@Mytherin in #20098 [Storage] Fix NULL filter check for constant segments by @@Tishj in #20103 fixing staged upload for install.duckdb.org - hopefully by @@hannes in #20104 Remove undefined loop by @@artjomPlaunov in #20105 [Test] Adjust concurrent attach-detach test to expect write-write conflict by @@taniabogatsch in #20108 Fix ALIAS function in filter pushdown to preserve column aliases(#20008) by @@henry8128 in #20106 Headers missing from http logs by @@samansmink in #20030 Bumping httpfs to include duckdb/duckdb-httpfs#174 by @@carlopi in #20145 Issue #20136: Secondary IGNORE NULLS by @@hawkfish in #20153 Fix minio nightly tests by @@c-herrewijn in #20132 Fix use-after-free in mode aggregate Combine function by @@victor-ab in #20146 Only pushdown varchar if the arrow type is not a string view by @@evertlammerts in #20165 Quote filters in adbc_get_objects by @@evertlammerts in #20172 Issue #20156: Streaming Window Unions by @@hawkfish in #20191 MergeInto: correctly clean-up buffer and handle non trivial GetData by @@carlopi in #20163 Fix extentension-ci-tools to latest release version in the v1.4-andium branch by @@carlopi in #20227 [Chore] Clean up CI nightly run to prevent time outs by @@taniabogatsch in #20150 Bump Julia to v1.4.3 by @@maiadegraaf in #20248 Internal #6881: 2025c Time Zones by @@hawkfish in #20258 Split statements by semicolon by @@Dtenwolde in #20174 [chore] Remove numeric_cast.hpp import and other tidy stuff by @@taniabogatsch in #20278 [chore] Reduce latency of VARCHAR index creation test by @@taniabogatsch in #20277 Reuse correct table-level metadata during checkpoints by @@ywelsch in #20267 Fix view resolution not being stable if the referenced table lives in a different schema by @@Tishj in #20260 ConstantOrNullFunction input validity mask overwrite bugfix by @@artjomPlaunov in #20283 dbgen: use TaskExecutor framework by @@Mytherin in #20284 fix(adbc): return error when setting an empty sql query by @@gishor in #20071 Bump iceberg, and add wasm platforms! by @@carlopi in #20205 _extension_distribution: Pin to v1.4-andium branch of extension-ci-tools by @@carlopi in #20294 Optimize prepared statement parameter lookups by @@EtgarDev in #20252 Update VectorType in ComputePartitionIndices by @@lnkuiper in #20343 Add some defensive programming in RadixPartitionedHashTable::Combine and RadixPartitionedHashTable::Finalize by @@lnkuiper in #20342 Increase reserved size for paths in SetPathsInternal by @@Flogex in #20340 Use UTF-16 console output in Windows shell (1.4) by @@staticlibs in #20339 Force repartitioning in RadixPartitionedHashTable::Combine by @@lnkuiper in #20357 Fixup comparison to wrong iterator on abdc's driver by @@carlopi in #20360 [chore] dsdgen generation signed integer overflow fix by @@taniabogatsch in #20279 Give preference do variables set in config by @@pdet in #20396 Add QueryContext also to Write(void *buffer, idx_t nr_bytes), so that BytesWritten are updated also there by @@carlopi in #20393 Fix Issue #20233: fix function chain in qualify by @@ArNine in #20302 Backport client data cleanup by @@taniabogatsch in #20403 Add v1.4.4 to Storage Version by @@maiadegraaf in #20404 Parquet Reader: Ignore invalid UTF8 in string stats, instead of throwing an error by @@Mytherin in #20405 Don't add a semicolon to final query when splitting statements by @@Dtenwolde in #20401 Fixup BRANCHES_TO_BE_CACHED, vars are not available on PRs, so env it is by @@carlopi in #20411 [Fix] Defensive infinite loop guard and UTF-8 check in C API by @@taniabogatsch in #20348 Add ducklake, httpfs and iceberg tests so they are run in CI by @@carlopi in #20319 [Stats] date_trunc stat propagation fix by @@Tishj in #20421 Refactor allowed path sanitize by @@hannes in #20346 Issue #20413: ASOF SEMI/ANTI Bindings by @@hawkfish in #20433 Fix #20410: fix for RIGHT SEMI/ANTI - cannot fully label chain as found if there are non-equality predicates present in the join condition by @@Mytherin in #20435 [Fix] Misaligned size in ART prefix count by @@taniabogatsch in #20455 Wrap ccache in own action by @@carlopi in #20466 [Chore] Add ORDER BY to AS OF test by @@taniabogatsch in #20465 Fix segfault in hive partitioning with NULL values by @@Schwarf in #20468 Nightly test encryption fixes by @@ccfelius in #20461 [C API] Fix error data creation by @@taniabogatsch in #20451 Skip concurrent_encrypted_attach due to race in autoloading httpfs by @@carlopi in #20471 CI fixup: Comparisons needs to be done via strings (since input via workflow_call is a string) by @@carlopi in #20464 Add unity_catalog to extensions built by duckdb/duckdb by @@carlopi in #20445 Invert the setup of ccache and cleanup_runner by @@carlopi in #20429 bump spatial for andium by @@Maxxen in #20479 Reclaim disk space on MacOS runners by @@lnkuiper in #20493 Adding secure clear functions by @@ccfelius in #20285 rewrite unaligned scan by @@artjomPlaunov in #20474 Issue #20470: TIMESTAMPTZ to DATE by @@hawkfish in #20498 Reset cached dictionaries in TRY expression by @@Maxxen in #20452 [Copy] Fix #20324 partition_by option binding by @@Tishj in #20509 Bump multiple extensions by @@maiadegraaf in #20504 update azure ref for v1.4-andium by @@benfleis in #20506 [Fix] Directly retrieve the logical column index during MERGE INTO binding by @@taniabogatsch in #20503 Sanitize pragmas by @@samansmink in #20514 Bump httpfs, ducklake and postgres by @@carlopi in #20518 Revert vortex bump for v1.4.4 by @@taniabogatsch in #20527 Bump VSS by @@taniabogatsch in #20542 Bump Excel by @@maiadegraaf in #20540 Fix Issue #20233: Fix function chain in having and merge to v1.4 by @@ArNine in #20532 bump iceberg by @@Tmonster in #20549 Bump Iceberg v1.4 by @@Tmonster in #20604 Fix KeyValueSecretReader init by @@NiclasHaderer in #20620 Fail fast extensions by @@carlopi in #20605 1.4.3: What's Changed implement proper equals for table functions by @@Maxxen in #19722 Fix behavior for HAVING clause without a GROUP BY by @@Tishj in #19739 Remove httpfs patches and bump by @@carlopi in #19763 Correctly use a lock when accessing the EncryptionKeyManager by @@Mytherin in #19772 Bump iceberg to now default available extension by @@carlopi in #19764 Extract all column bindings of json each function by @@Tmonster in #19766 constraint violation bug fix by @@artjomPlaunov in #19776 Avoid binding macro arguments for untyped macros by @@lnkuiper in #19779 It should probably be ${EXTENSION_CONFIG_BUILD} instead of EXTENSION_CONFIG_BUILD by @@hannes in #19694 Fix #19575: Fix illegal utf8 character by @@flashmouse in #19699 chore: bump vortex to 0.56.0 by @@0ax1 in #19815 PositionalScan: handle also HAVE_MORE_OUTPUT + empty chunk via iteration by @@carlopi in #19824 AttachDatabase: first set whether file is remote, then do the rest by @@carlopi in #19826 Add CacheBehavior::AUTOMATIC to DBInstanceCache that automatically does the right thing™ by @@Mytherin in #19841 Reuse metadata even in the presence of deletes by @@ywelsch in #19823 DuckIndexScanState::TableScanFunc, split into 2 explicit phases by @@carlopi in #19838 Bump httpfs and iceberg by @@carlopi in #19859 Keep cte_root alive while binding materialized CTEs in MERGE INTO children by @@Mytherin in #19863 CI Cleanup (#19840) by @@yan-alex in #19857 Fix #19517: preserve relation name for table-qualified star LIKE expression by @@henry8128 in #19887 avoid underflow/"inf" loop while reporting unittest summary (backport… by @@benfleis in #19900 Fixes incorrect handing of APPROX_QUANTILE TIME by @@Damon07 in #19891 Add v1.4.3 to Storage Version by @@maiadegraaf in #19907 clean up tmp files while building extensions by @@c-herrewijn in #19908 Bump: ducklake, mysql_scanner by @@maiadegraaf in #19910 remove large limit optimization whenever there is a filter by @@guillesd in #19911 free disk space in Upload Extensions job by @@c-herrewijn in #19912 [Compression] Prevent overriding COMPRESSION_EMPTY with COMPRESSION_CONSTANT by @@Tishj in #19913 Make make tidy-check-diff compare against base branch, instead of always comparing against origin/main by @@Mytherin in #19917 More testing for appender and attach-detach by @@taniabogatsch in #19708 Script to release a extension manually by @@samansmink in #19881 Assert that result types match with column data collection when fetching data by @@pdet in #19750 No longer auto-enable profiling on DEBUG mode by @@maiadegraaf in #19931 Fix optimizer incorrectly remove ORDER BY clause from aggregates by @@tianjq16 in #19925 Backport revert append fixes by @@Mytherin in #19941 Bump: spatial by @@Maxxen in #19943 Issue #19916: WASM Time Zones by @@hawkfish in #19918 Fix correlated column binding in ConstantBinder by @@d-justen in #19945 [chore] Increase slow threshold by @@taniabogatsch in #19965 [Parquet] Fix prepared copy option parameter by @@Tishj in #19966 [TestConfig] Fix verify_fetch_row config, reduce duplication in skip_tests by @@Tishj in #19967 remove sha from artifacts by @@c-herrewijn in #19957 Unbound index binding with context by @@artjomPlaunov in #19953 [Fix] Bug in FetchRow after update on indexed table with dict_fsst compression by @@taniabogatsch in #19970 Null assertion on denormalized_table argument by @@Dtenwolde in #19947 [Art][Wal]Unbound index allocations by @@artjomPlaunov in #19901 No sampling over 281TB by @@Tmonster in #19978 Take in consideration if the sniffer used unstrictness while selecting candidates by @@pdet in #20005 [CSV] Avoid throwing unnecessary errors on strict mode by @@pdet in #20007 [Compression] Fix an issue with NULL updates to a column compressed with DICT_FSST by @@Tishj in #20009 Bump: delta, ducklake, iceberg by @@maiadegraaf in #20012 add unity_catalog, update delta by @@samansmink in #20019 Bump spatial by @@staticlibs in #20020 Retag #19821 to v1.4 - Use PLAIN_DICTIONARY for Parquet version 1 by @@pdet in #20024 Fix INSERT OR REPLACE BY NAME with partial columns(#19845) by @@henry8128 in #19989 Bump MySQL scanner by @@staticlibs in #20025 Windows must sample less by @@Tmonster in #20018 Issue #20015: Streaming Window Sequence by @@hawkfish in #20027 bump iceberg by @@Tmonster in #20032 Fix #20014: correctly use numeric_limits::min for NumericLimits::Min so that stats are initialized to -infinity for floating points by @@Mytherin in #20039 Enable windows_arm64 arch for main extensions by @@staticlibs in #20004 Fix mark join decorrelation by @@kryonix in #20033 propagate gate status in Node4::DeleteChild by @@artjomPlaunov in #20044 Bump: httpfs by @@samansmink in #20036 Bump ducklake by @@pdet in #20054 bump azure to v1.4.3 by @@benfleis in #20057 Bump extensions by @@samansmink in #20055 Fix unnecessary dependent join rewrite by @@kryonix in #20048 Version tag for local files for robust external file cache validation by @@lnkuiper in #20058 backport runner cleanup action by @@hannes in #20085 1.4.2: What's Changed bump duckdb-azure ref for 1.4.1 by @@benfleis in #19275 Fix regex optimization to remove flags when converting to contains by @@mlafeldt in #19290 feat: Fix length of sort keys by @@krlmlr in #19260 Pass lambda bindings to next binder by @@Dtenwolde in #19144 bump Julia to v1.4.1 by @@c-herrewijn in #19306 Use cross-compilation for static libs on OSX by @@hannes in #19304 Allow multiple read-only attaches to the same database across database instances by @@Mytherin in #19319 hugeint_t fixes by @@carlopi in #19318 Use CMAKE__COMPILER_LAUNCHER by @@evertlammerts in #19326 Add settings field to test config by @@Mytherin in #19330 Keep track of which database managers have which databases attached in the DatabaseFilePathManager by @@Mytherin in #19338 Bugfixes by @@lnkuiper in #19329 skip several ci jobs on prs that only bump extensions by @@samansmink in #19249 [C API] bind_value out of range fix by @@taniabogatsch in #19348 Destroy TaskNotifier prior to calling FinishTask by @@Mytherin in #19373 Remove zip bombs by @@Mytherin in #19380 Avoid calling shared_from_this() and instead use the passed in ClientContext in buffered data by @@Mytherin in #19379 add test tag support [vfs integration tests p1] by @@benfleis in #19331 Throw if non-VARCHAR key is passed to json_object by @@lnkuiper in #19365 BUGFIX: Silent failure to write row groups with large lists by @@J-Meyers in #19376 Fixes for CTE (de)serialization compatibility with older versions by @@Mytherin in #19393 Fix #18139: correctly initialize flush size in MemoryStream, and re-use writer states by @@Mytherin in #19398 Fixup Kalman fiter: bound it's (0, scale_factor), not (0, 1) by @@carlopi in #19395 Fix StringDecompress for hugeint_t values by @@ywelsch in #19403 Fix #19211: make INSERT OR IGNORE correctly handle multiple constraints by @@Mytherin in #19409 Issue #19386: ICU TZDEFAULT by @@hawkfish in #19413 Fix issue in MetadataManager triggered when doing concurrent reads while checkpointing, and rework concurrent attach / detach test by @@Mytherin in #19424 ADBC fix: escape schema, table and column identifiers by @@evertlammerts in #19407 v1.4: Only serialize CTE nodes when MATERIALIZED is specified by @@Mytherin in #19420 Add forwards compatibility tests to CI by @@Mytherin in #19432 Internal #6168: Unsupported Correlated Binds by @@hawkfish in #19431 Don't pull up filters through DISTINCT ON and make enum casts in Parquet safe (bugfixes) by @@lnkuiper in #19406 Get table bindings for all operators under a logical get if the logical get function is an unnest by @@Tmonster in #19467 WASM #1897: UTC Offset Support by @@hawkfish in #19464 Add tests for nested lambda bindings by @@Dtenwolde in #19453 [Compression] Fix issue in ZSTD decompression related to mis-interpreted segment offsets by @@Tishj in #19475 vfs integration tests p2 - update (env) vars and tests by @@benfleis in #19428 CI Fix: clear benchmark cache between runs by @@Mytherin in #19508 Make DatabaseInstance::log_manager a unique_ptr by @@Flogex in #19471 Support non-standard NULL in Parquet again by @@Mytherin in #19523 Bump: inet by @@samansmink in #19526 Bump multiple extensions by @@samansmink in #19522 add upcoming patch release to internal versions by @@samansmink in #19525 Add test that either 'latest' or 'vX.Y.Z' are supported STORAGE_VERSIONs by @@carlopi in #19527 Moving staging to cf and uploading to install bucket by @@hannes in #19539 Creating separate OSX cli binaries for each arch by @@hannes in #19538 [Dev] Disable the use of ZSTD if the block_manager is the InMemoryBlockManager by @@Tishj in #19543 Follow up to staging move by @@hannes in #19551 Bugfixes: Parquet JSON+DELTA_LENGTH_BYTE_ARRAY and sorting iterator by @@lnkuiper in #19556 [ported from main] Fix bug initializing std::vector for column names by @@evertlammerts in #19555 Disable jemalloc on BSD by @@lnkuiper in #19560 Fix race condition between Append and Scan by @@Captain32 in #19571 Release relevant tests to still be run on all builds by @@carlopi in #19559 fix inconsistent behavior in remote read_file/blob, and prevent union… by @@Maxxen in #19531 [v1.4-andium] Add Profiler output to logger interface by @@carlopi in #19572 Fix edge case in uncompressed validity scan with offset and fix off-by-one in ArrayColumnData::Select by @@Maxxen in #19567 Skip compiling remote optimizer test when TSAN Is enabled by @@Mytherin in #19590 Wal index deletes by @@artjomPlaunov in #19477 add vortex external extension by @@samansmink in #19580 Avoid eagerly resolving the next on-disk pointer in the MetadataReader, as that pointer might not always be valid by @@Mytherin in #19588 [DevEx] Improve error message when FROM clause is omitted by @@Tishj in #18995 bump iceberg by @@Tmonster in #19618 Improve error message around compression type deprecation/availability checks by @@Tishj in #19619 Increase cast-cost of old-style implicit cast to string by @@Maxxen in #19621 Try to prevent overshooting of FILE_SIZE_BYTES by pre-emptively increasing bytes written in Parquet writer by @@lnkuiper in #19622 Bump: spatial by @@Maxxen in #19620 Detect invalid merge into action and throw exception by @@Mytherin in #19636 Fix #19455: correctly extract root table in merge into when running ajoin that contains single-sided predicates that are transformed into filters by @@Mytherin in #19637 Remove FlushAll from DETACH by @@lnkuiper in #19644 Bump MySQL scanner by @@staticlibs in #19643 Enable running all extensions tests as part of the build step by @@carlopi in #19631 Always remember extra_metadata_blocks when checkpointing by @@ywelsch in #19639 duckdb_logs_parsed to do case-insensitive matching by @@carlopi in #19669 Categorize ParseLogMessage as CAN_THROW_RUNTIME_ERROR LVM by @@carlopi in #19668 Log total probe matches in hash join by @@lnkuiper in #19683 Fix InsertRelation on attached database by @@evertlammerts in #19583 Add request timing to HTTP log by @@samansmink in #19691 Add missing query location tohttpfs by @@samansmink in #19715 Fix #19355: correctly resolve subquery in MERGE INTO action condition by @@Mytherin in #19720 Fix #19700: correctly sort output selection vector in nested selection operations by @@Mytherin in #19718 Bump httpfs and resume testing on Windows by @@carlopi in #19714 [Dev] Fix assertion failure for empty ColumnData serialization by @@Tishj in #19713 Bump the Postgres scanner extension by @@Mytherin in #19730 Add explicit Initialize(HTTPParam&) method to HTTPClient by @@carlopi in #19723 Logs to be case-insensitive also at enable_logging callsite by @@carlopi in #19734 Fix CVE GHSA-vmp8-hg63-v2hp crypto issues by @@samansmink in #19716 1.4.1: What's Changed Fix attach to right DB when using DuckLake by @@pdet in #19011 set default value of MAIN_BRANCH_VERSIONING to false by @@c-herrewijn in #19014 ComplexJSON: parse all valid JSON correctly by @@Mytherin in #19024 Issue #19016: ICU Offset Parsing by @@hawkfish in #19029 Throw if we detect a quoted new line with the null padding set in parallel mode by @@pdet in #19012 Bump iceberg & ducklake by @@carlopi in #19037 Build Fix: unordered_map https://blobs.duckdb.org by @@carlopi in #19206 [chore] Attempt at restoring workflow for MinGW Static libs by @@carlopi in #19205 Simple no default region return 301 response by @@Tmonster in #19087 [Fix] Correctly reset the gate status during ART merging by @@taniabogatsch in #19204 build spatial extension for mingw by @@c-herrewijn in #19207 Fixup templated version of TryGetSecretKeyOrSetting by @@carlopi in #19218 Bump: delta by @@samansmink in #19220 Autoloading helper file system: allow either autoloading or proper errors in more file operations by @@carlopi in #19198 Eargerly destroy sort buffers in Window by @@lnkuiper in #19224 [Transaction] Delete and drop of a table can now happen in the same transaction without error by @@Tishj in #18918 PRAGMA's MissingEntry: Suggest CALL might be an option by @@carlopi in #18815 Bump: aws, ducklake, iceberg by @@samansmink in #19228 Issue 18603 by @@Tmonster in #19227 Bump DuckLake to latest of V1.4 by @@pdet in #19237 Bump mysql and sqlite by @@staticlibs in #19240 Don't write parquet-native GEOMETRY by default, add option to control GeoParquet version by @@Maxxen in #19244 When executing a relation, generate a query to set if it is not a query relation by @@Mytherin in #19234 add support for writing geoparquet with v2 metadata too by @@Maxxen in #19246 Bump: iceberg by @@samansmink in #19250 Bump: avro, httpfs by @@samansmink in #19248 bump duckdb-azure ref for 1.4.1 by @@benfleis in #19275 1.4.0: What's Changed Python package devexp improvements by @@evertlammerts in #17483 change exception type to not be an internal exception by @@samansmink in #17551 Remove redundant code path in the ConflictManager by @@taniabogatsch in #17562 Add support for ToSqlString for union types by @@wmTJc9IK0Q in #17513 Update function descriptions and examples by @@c-herrewijn in #17132 Move query profiler's EndQuery after commit/rollback by @@taniabogatsch in #17595 fix extension troubleshooting link by @@simon0191 in #17616 C API tidying by @@taniabogatsch in #17623 bump DuckDB_jll to v1.3.0 by @@c-herrewijn in #17677 Add rowsort in generate_series test #43 by @@jeewonhh in #17675 [C API] Expose duckdb_scalar_function_bind_get_extra_info by @@taniabogatsch in #17666 Enable profiling output for all operator types by @@taniabogatsch in #17665 Output hashes in unittest and fix order by @@niykko in #17664 New Sorting Implementation by @@lnkuiper in #17584 Merge v1.3-ossivalis into main by @@Mytherin in #17690 Issue #17040: FILL Window Function by @@hawkfish in #17686 ClientBufferManager wrapper to access the client context in the buffer manager by @@taniabogatsch in #17699 Revert "set default for MAIN_BRANCH_VERSIONING to false" by @@carlopi in #17708 Sorting followup by @@lnkuiper in #17717 Correctly setting the delim offset by @@Damon07 in #17716 fix linux extension ci by @@samansmink in #17720 Aggregation performance by @@lnkuiper in #17718 Fix windows-2025 build errors by @@adsharma in #17726 [SQLLogicTester] Introduce reset label in the tester by @@Tishj in #17729 Adding additional authenticated data for encryption by @@ccfelius in #17508 csv_scanner: correct code comment by @@Djfe in #17735 Deprecate windows-2019 runners by @@hannes in #17745 re-add httpfs apply_patches by @@samansmink in #17755 Rename decorator from test_nulls to null_test_parameters by @@Mytherin in #17760 [CAPI] Expose ErrorData by @@taniabogatsch in #17722 Expose file_size_bytes and footer_size in parquet_file_metadata by @@gijshendriksen in #17750 Pass ExtensionLoader when loading extensions, change extension entry function by @@Maxxen in #17772 Support glibc 2.28 environments by @@James-Gilbert- in #17776 Mark Upper/LowerComparisonType as const by @@JelteF in #17773 [Indexes] Buffer-managed indexes part 1: segment handles by @@taniabogatsch in #17758 [Julia] api docs improvements by @@tqml in #15645 Ensure we use the same layout in RadixPartitionedHashTable and GroupedAggregateHashTable by @@lnkuiper in #17790 [Profiling] Propagate the ClientContext into file handle writes by @@taniabogatsch in #17754 Fix propagatesNullValues for case expr by @@suibianwanwank in #17796 Add qualified parameter to Python GetTableNames API by @@evertlammerts in #17797 Merge v1.3 into main by @@Mytherin in #17806 Pushdown pivot filter by @@flashmouse in #17801 Replace string for const data ptr in encryption api by @@ccfelius in #17825 Merge130 by @@carlopi in #17833 fix: escape using_columns on JoinRef::ToString by @@akoshchiy in #17839 Fix ICE with Windows ARM64 by @@staticlibs in #17844 Merge v1.3 into main by @@Mytherin in #17851 Add duckdb_type column to parquet_schema by @@Mytherin in #17852 Internal #4991: Remove Epoch_MS(MS) by @@hawkfish in #17816 #17853 Enable flexible page sizes and update Android NDK to r27 in workflow. by @@aprock in #17854 [Indexes] Buffer-managed indexes part 2: segment handle for base nodes by @@taniabogatsch in #17828 Function Serialization: adapt to removal of overloads by explicitly casting if argument types have changed by @@Mytherin in #17864 julia: add missing methods from C-API by @@tqml in #17733 Issue #17153: Window Order Columns by @@hawkfish in #17835 Issue #17040: FILL Secondary Sorts by @@hawkfish in #17821 Add STRUCT to MAP cast function by @@evertlammerts in #17799 Issue #17849: Test FILL Duplicates by @@hawkfish in #17869 Add GenAI policy by @@szarnyasg in #17882 Update function descriptions and examples for list, array, lambda functions by @@c-herrewijn in #17886 Issue #17861: FILL Argument Types by @@hawkfish in #17888 Reword GenAI policy by @@szarnyasg in #17895 Use an arena linked list for the physical operator children by @@taniabogatsch in #17748 Make CTE Materialization the Default Instead of Inlining by @@kryonix in #17459 Merge v1.3 into main by @@Mytherin in #17897 Leverage VectorType in ColumnDataCollection by @@lnkuiper in #17881 Fix empty BP block when writing parquet by @@platypii in #17929 fix use after free in adbc on invalid stmt by @@ruslandoga in #17927 Do not dispatch JDBC/ODBC jobs in release CI runs by @@staticlibs in #17937 Block based encryption by @@ccfelius in #17275 Unittester failures summary by @@hmeriann in #16833 Add v1.3-ossivalis to Cross version workflow by @@hmeriann in #17906 [CI Nightly Fix] Skip logging test if not standard block size by @@taniabogatsch in #17957 Visual Studio 17 (2022) fixes by @@edouarda in #17948 [Nested] Add struct_position and struct_contains functions by @@maiadegraaf in #17819 Enable building spatial and encodings extensions by @@staticlibs in #17960 [Nested] Optimize structs in LIST_by @@hmeriann in #17949 Issue #5144: AsOf Join Threshold by @@hawkfish in #17979 [Fix] Reset profiling info before preparing a query by @@taniabogatsch in #17940 Flag to disable database invalidation by @@taniabogatsch in #17938 Issue #5123: mpdet in #17947 c-api to copy vector with selection by @@abramk in #17870 Fix #18007: correctly execute expressions with pivot operator by @@Mytherin in #18020 [Chore] Minor conflict manager refactoring by @@taniabogatsch in #18015 Remove LinuAML format for EXPLAIN statements by @@qsliu2017 in #17572 Unittest: Add skip_compiled option that can be used to skip built-in C++ tests by @@Mytherin in #18034 Add ppc64le spin-wait instruction by @@mgiessing in #17837 Merge ossivalis into main by @@taniabogatsch in #18036 Remove match-case statements from polars_io.py by @@evertlammerts in #18052 Avoid adding commands read from a file to the shell history by @@Mytherin in #18057 Adding WAL encryption by @@ccfelius in #17955 Encryption: adding -key for the command line by @@ccfelius in #17950 fix star expr exclude error by @@jayhan94 in #18063 Add support for class-based expression iteration by @@Mytherin in #18070 Use timestamp_t instead of time_t for file last modified time by @@lnkuiper in #18037 Unittester: add on_new_connection + on_load + skip_tests options by @@carlopi in #18042 Fix some scaling issues by @@lnkuiper in #17985 Issue #18071: Temporal inf -inf by @@hawkfish in #18083 Switch to Optional for type hints in polars lazy dataframe function by @@evertlammerts in #18078 Unittest: Configure skip error messages by @@carlopi in #18087 Avoid running DraftPR.yml until timeout if token is missing by @@carlopi in #18090 Add start/end offset percentage options to Python test runner by @@Flogex in #18091 [CSV Reader] Prohibit options delim and sep in same read_csv call by @@ackxolotl in #18096 Fix correlated subquery unnest fail by @@flashmouse in #18092 [CI] don't run jobs on draft PRs by @@hmeriann in #18016 TPC-DS: Use BIGINT fields by @@szarnyasg in #18098 Don't throw InternalException in Sort::Sink by @@lnkuiper in #18105 ci: build duckdb against the latest emscripten by @@cpcloud in #18110 [chore] Merge v1.3-ossivalis on main by @@carlopi in #18109 Update description of 'arrow_lossless_conversion' by @@szarnyasg in #18046 Internal #3273: Window Task Generation by @@hawkfish in #18113 set ::error:: annotations for test runners by @@hmeriann in #18072 Improve sort key comparison performance by @@lnkuiper in #18131 Add support for MERGE INTO by @@Mytherin in #18135 Detect when updates have no effect, and skip performing the actual updates if we encounter these nop updates by @@Mytherin in #18144 Add support for AdbcConnectionGetObjects(table_type) by @@kou in #18066 Issue #17683: TIME_NS Compilation by @@hawkfish in #18053 Implement replace_type function by @@lnkuiper in #18077 Bump spatial again: include re-linking for handling global objects in Wasm by @@carlopi in #18170 Resolve some small build issues by @@madscientist in #18162 fix typo by @@felixhummel in #18165 Avoid realloc in CSV writer by @@lnkuiper in #18174 fix bug with allowed_paths by @@samansmink in #18176 Reduce lock contention for the instance cache by @@NiclasHaderer in #18079 Check if GetLastSegment is not nullptr in ColumnData::RevertAppend by @@lnkuiper in #18171 [Profiling] Move the client context into more write functions by @@taniabogatsch in #17875 Bump Julia to v1.3.2 by @@hmeriann in #18185 Merge v1.3-ossivalis into main by @@carlopi in #18188 Parquet reader logging by @@lnkuiper in #18172 Add VS2019 compat flag to Python wheel build by @@staticlibs in #18198 [Parquet][Dev] Update the vendored parquet.thrift to 3ce0760 by @@Tishj in #18195 Two-rowID-leaf support in the conflict manager and general refactoring by @@taniabogatsch in #18194 More internal-linkage by @@Maxxen in #18177 Temporary file encryption by @@Mytherin in #18208 Adding temporary file encryption by @@ccfelius in #18013 Skip logging test for smaller block sizes by @@taniabogatsch in #18201 ci(pyodide): enable WASM exceptions on the latest pyodide build by @@cpcloud in #18173 Allow explicit compression for user types by @@lnkuiper in #18219 Get type of encoded SortKey from TupleDataLayout by @@lnkuiper in #18218 Improve Parquet reader NULL statistics and compress all-NULL columns using CompressedMaterialization by @@lnkuiper in #18217 Internal #5264: NLJ Not Distinct by @@hawkfish in #18216 Bug#18163 Fix STDDEV_SAMP undeterminism by @@minaracic in #18210 [Parquet] Add read support for the VARIANT LogicalType by @@Tishj in #18187 Track DataChunk memory usage in various places by @@lnkuiper in #18191 Better NULL handling in TupleDataLayout by @@lnkuiper in #18069 Dictionary functions by @@lnkuiper in #18127 Add support for geoarrow encoded geometries in geoparquet files. by @@cfis in #17942 Improve descriptions of thresholds vars affecting join algorithm selection by @@TheHillBright in #17377 Connect relations that are not in a subgraph, but are still part of the new relation set by @@Tmonster in #18182 [Fix] Don't write empty (partial) blocks for FSST dictionary compression by @@taniabogatsch in #18203 Slightly higher memory limit for test by @@lnkuiper in #18235 Re-add string -> hugeint compressed materialization function by @@lnkuiper in #18234 [Fix] Database path conflict resolution by @@taniabogatsch in #18247 Remove require block size from a batch of tests by @@taniabogatsch in #18242 Add nightly builds for out-of-tree python extension by @@evertlammerts in #18239 Backport DB invalidation flag to ossivalis by @@taniabogatsch in #18244 Bump vcpkg-duckdb-ports and test extensions on Windows 10 default stdlib by @@carlopi in #18205 Add type safety to FlatVector::GetData, ConstantVector::GetData and UnifiedVectorFormat::GetData by @@Mytherin in #18256 [Fix] Adjust test for smaller block sizes by @@taniabogatsch in #18255 Fix integer overflow in sequence vector by @@xuke-hat in #18245 fixes for some minor llvm 20 complaints by @@hannes in #18257 update run_extension_medata_tests.sh by @@hmeriann in #17976 Bunch of loosely connected test/CI fixes by @@carlopi in #18254 disable WebAssembly duckdb-wasm builds job in NightlyTests triggered by 'workflow_dispatch' event by @@hmeriann in #18129 Allow for static libs from extension dependencies to be bundled by @@abramk in #18226 Fix dictionary-related assertions by @@lnkuiper in #18260 Fixes for gcc 15 by @@hannes in #18261 Reduce copy in Vector::Reinterpret by @@xuke-hat in #18264 [Parquet] Add read support for the VARIANT LogicalType (with shredded encoding) by @@Tishj in #18224 Expanded autocomplete suggestions by @@Dtenwolde in #18243 Support HUGEINT in printf and format by @@xuke-hat in #13277 Move aarch64 / arm64 to native github runner by @@evertlammerts in #18269 Bump vcpkg-duckdb-ports to solve OSX linking by @@carlopi in #18268 Add support for RETURNING to MERGE INTO by @@Mytherin in #18271 Use set for row ID scanning during index scans by @@taniabogatsch in #18274 Use DuckDB cast infrastructure in fmt for new uhugeint/hugeint code by @@Mytherin in #18275 [Fix] Adjust test to run with different block sizes by @@taniabogatsch in #18277 Use FromEpochSeconds instead of FromTimeT in FileSystem::GetLastModifiedTime by @@lnkuiper in #18281 Add target for installing Python deps. by @@xevix in #18285 backport 'Unit Tester Configuration' pt2 by @@c-herrewijn in #18282 backport 'Unit Tester Configuration' by @@c-herrewijn in #18263 Fixup Main.yml for v1.3-ossivalis post #18282 by @@carlopi in #18289 SHOW TABLES FROM by @@xevix in #18179 [Unittester] Add autoloading option by @@carlopi in #18290 Merge ossivalis into main by @@taniabogatsch in #18272 resolve hidden merge conflict with duplicate db name in json configs by @@c-herrewijn in #18292 Bump vcpkg-duckdb-ports, now fixing also mingw by @@carlopi in #18300 [Fix] Missing block when renaming fields by @@taniabogatsch in #18293 [Arrow] Fix unused static function warning by @@Tishj in #18278 feat: Parquet extension add row_group_compressed_size by @@mapleFU in #18294 [Parquet][Write] Fix timestamp sec writes to parquet by @@Tishj in #18273 bump httpfs by @@Tmonster in #18258 [Clang Tidy] Fix missing includes in patas_scan.hpp by @@Tishj in #18276 New Arrow C-API by @@pdet in #18246 Skip test/sql/copy/s3/url_encode.test due to httpfs update by @@carlopi in #18317 Make storage-version a test parameter by @@Mytherin in #18324 Backport #18254 by @@carlopi in #18306 feat: making Parquet write RowGroup.total_compressed_size by @@mapleFU in #18307 add the from-table-function as parameter to copy-from-bind by @@peterboncz in #18004 Python external dispatch param fixes by @@evertlammerts in #18343 Aarch64 backport by @@evertlammerts in #18345 Fix debug error in join order optimizer by @@Tmonster in #18344 [Fix] Block verification for add and drop field info by @@taniabogatsch in #18302 download Real Nest data in quiet mode by @@hmeriann in #18346 Fix condition indexes in join filter pushdown by @@Damon07 in #18341 [unittest] - fix doubled error headers on Unexpected failure by @@hmeriann in #18314 Extend PEG parser grammar by @@Dtenwolde in #18221 [C API] Expose expressions and use them in scalar function binding by @@taniabogatsch in #18142 Restore OSX tests, moving them to single --autoloading available step by @@carlopi in #18335 Add support for checkpointing in-memory tables by @@Mytherin in #18348 Revert "[unittest] - fix doubled error headers on Unexpected failure" by @@Mytherin in #18355 Python external dispatch param fixes by @@evertlammerts in #18359 Re-enable url-encode test by @@Tmonster in #18360 Enable stack traces on linux for bundled libraries by @@taniabogatsch in #18363 Split up out-of-tree extensions into separate files, and allow out-of-tree extensions to be built using BUILD_EXTENSIONS={ext_name} by @@Mytherin in #18357 Pass AttachOptions to attach method, and turn StorageExtensionInfo into an optional_ptr by @@Mytherin in #18368 Merge v1.3-ossivalis into main by @@carlopi in #18364 More robustness around deprecated extension settings by @@carlopi in #18353 Add missing ninja to workflow file by @@Dtenwolde in #18373 bump httpfs by @@Tmonster in #18380 Re-enable but deprecate CORE_EXTENSIONS in CMakeLists.txt by @@evertlammerts in #18377 Uncomment skipped decimal REE tests by @@amoeba in #18372 add option 'block_size' to test configs by @@c-herrewijn in #18347 [chore] Fixup side-effects from 8cf9ed4 by @@carlopi in #18385 Bump httpfs by @@taniabogatsch in #18388 Re-use table metadata when table is not altered during checkpoint by @@Mytherin in #18390 Approx database count system function by @@taniabogatsch in #18392 Re-use metadata of unaltered row groups when checkpointing a table by @@Mytherin in #18395 Correct and consistent integer arithmetic error messages by @@soerenwolfers in #18393 Record whether or not cross products are implicit or not, and use this for converting queries back to SQL by @@Mytherin in #18394 CI: Fix Discussion mirroring by @@szarnyasg in #18397 Store extra metadata blocks in RowGroupPointer, and only flush dirty Metadata blocks by @@Mytherin in #18398 Internal #3273: Window Hashed Sort by @@hawkfish in #18337 Wrap runner.ExecuteFile, otherwise cleanup is not properly performed by @@carlopi in #18400 [BUGFIX] Update delim offset for RHS of DELIM JOIN when correlated column is in RHS of Cross product by @@Tmonster in #18375 CI: Add separate job for discussion mirroring by @@szarnyasg in #18407 [ Python SQLLogic Tester ] Add MERGE_INTO statement to duckdb python by @@hmeriann in #18402 Remove incorrect assertion by @@Mytherin in #18404 Internal #5294: TIME_NS C API by @@hawkfish in #18215 Add DuckLake back in by @@Mytherin in #18405 Add support for table_constraints of AdbcConnectionGetObjects() by @@kou in #18181 Merge v1.3-ossivalis in main by @@carlopi in #18401 feat: remove anything following ? in database name by @@rustyconover in #18417 Correctly fetch only base column data in ColumnData::FetchUpdateData by @@Mytherin in #18423 Refactor extension CI to use extension-ci-tools by @@samansmink in #18361 Internal #5367: SortedAggregateFunction Sort Update by @@hawkfish in #18408 Internal #5368: WindowNaiveAggregator Sort Update by @@hawkfish in #18409 [Fix] Block size nightly by @@taniabogatsch in #18425 [Chore] Tidy test configs by @@taniabogatsch in #18426 Include pyodide build configuration by @@rgbkrk in #18183 Parquet: add row-group ordinal during writing encryption by @@mapleFU in #18433 [Fix] Reset segment memory when initialising new Prefix by @@taniabogatsch in #18441 Update pyodide build to 0.28.0 by @@rgbkrk in #18446 Add support for "template" types by @@Maxxen in #18410 Internal #5384: WindowDistinctAggregator Sort Update by @@hawkfish in #18442 [Chore] Improve skipped tests in test config and add verify_fetch_row config by @@taniabogatsch in #18436 Buffer index appends during WAL replay by @@taniabogatsch in #18313 Add support for generic settings, and move many settings over to generic settings by @@Mytherin in #18447 Internal #5385: WindowMergeSortTree Sort Update by @@hawkfish in #18461 Bump postgres to latest main by @@Mytherin in #18464 Merge ossivalis by @@taniabogatsch in #18456 Internal #5366: WindowDeltaScanner by @@hawkfish in #18468 SUM and + Operator for Varints by @@pdet in #18424 [Fix] Rework transaction logic in commit, rollback and checkpoint paths by @@taniabogatsch in #18474 re-nable extensions in invokeci by @@samansmink in #18476 Internal #5384: Window Sorting Polish by @@hawkfish in #18484 Unify ON CONFLICT and MERGE INTO by @@Mytherin in #18480 More insights around dict_fsst compression failure by @@taniabogatsch in #18475 Change ctrl-a/ctrl-e to move to start/end of line, not buffer by @@tpot in #18490 add delta linux back to ci by @@samansmink in #18491 Fix accidental internal exception in type transformation by @@hannes in #18492 [Profiling] Add client context into read functions by @@maiadegraaf in #18438 julia: config improvements by @@aplavin in #17585 fix: add missing space in AttachInfo::ToString() by @@rustyconover in #18500 Merge ossivalis by @@taniabogatsch in #18502 Change UNICODE to UTF8 by @@sheldonrobinson in #17586 Fix: Remove overly strict assertion on empty string value by @@Dtenwolde in #18504 Fix several bugs/fuzzer issues by @@Mytherin in #18503 Allow expressions to be used in ATTACH / COPY options by @@Mytherin in #18515 Remove immediate_transaction_mode from DB config options by @@jeewonhh in #18516 Temporarily excluding Build Pyodide wheel for Python 3.11 because it fails to build WASM wheels by @@hmeriann in #18508 ParserException for Pragma with named parameters by @@Dtenwolde in #18506 Add verify fetch row config to Main.yml by @@taniabogatsch in #18478 Adding WITH ORDINALITY to DuckDB by @@niykko in #16581 When tracking evicted_data_per_tag, track actual size on disk after temp file compression by @@Mytherin in #18521 Fix: Write the salt together with the HT offset when determining the value for key comparison by @@gropaul in #18374 Fix incorrect character encoding in GetLastErrorAsString on Windows by @@soutong in #18431 Dynamically determine dictionary size limit in Parquet writer (if unset) by @@lnkuiper in #18356 Internal #16560: Numeric TRUNC Precision by @@hawkfish in #18511 Consistently detect JSON schema indepent of number of threads by @@lnkuiper in #18522 ALP test: skip TPC-DS 67 - it is not consistent with floating point numbers by @@Mytherin in #18528 [Varint] Negation, Subtraction and Over/under-flow checking by @@pdet in #18477 fix: support both field orders for variant struct by @@samansmink in #18532 Add CAPI to retrieve client context for table functions by @@VGSML in #18520 Add StatementVerifier for EXPLAIN by @@lnkuiper in #18529 Use global index, not local id when creating filters in MultiFileColumnMapper by @@lnkuiper in #18537 Add support for explicit clean-up routine in test config, and exit multi-statement execution when an error is encountered by @@Mytherin in #18539 fix: improve handling variant nulls and nested types by @@samansmink in #18538 Allow overriding openssl version for FIPS compliance by @@abramk in #18499 Unittester: Add the --sort-style parameter that allows a fallback comparison where results are sorted according to a given sort-style by @@Mytherin in #18542 Restore missing test/configs/small_block_size.json file by @@hmeriann in #18507 [Fix] Follow-up PR to only delete unique row IDs by @@taniabogatsch in #18545 [ART] Node::Free refactoring by @@taniabogatsch in #18544 Implement special-case VARCHAR to JSON[] casts and vice versa by @@lnkuiper in #18541 Check if heap_block_ids is empty before getting start/end when destroying chunks in TupleDataCollection by @@lnkuiper in #18556 optimize/parquet: generate movable types for parquet by @@mapleFU in #18510 [easy] [no-op] Minor optimization on iterator lookup by @@dentiny in #15349 Fixing compilation with -std=cpp23 by @@hannes in #18557 Add compile option standalone-debug for clang by @@flashmouse in #17433 Rename the Varint type to Bignum by @@pdet in #18547 [Indexes] Buffer-managed indexes part 3: segment handle for Node48 and Node256 by @@taniabogatsch in #18567 fix: add formatting to explain row counts by @@rustyconover in #18566 [CSV Sniffer] Fixing bug of not properly setting skipped rows from sniffer by @@pdet in #18555 [Fix] Tidy check ossivalis by @@taniabogatsch in #18583 [Fix] Adjust shrink threshold back to original count > SHRINK_THRESHOLD by @@taniabogatsch in #18582 Flip left/right delim join based on cardinalities by @@lnkuiper in #18552 fix: use thousands separator and decimal for row counts induckbox output format by @@rustyconover in #18564 Force LIST/ARRAY child vectors on a Parquet single page by @@lnkuiper in #18578 String dictionary hash cache by @@lnkuiper in #18580 fix: libduckdb.so missing soversion by @@strophy in #18305 Pushdown filters on coalesced outer join keys compared for equality under the join condition by @@matteobilardi in #18169 Adds a function for updating and adding values in a struct by @@teaguesterling in #15533 fix hidden merge conflict by @@hannes in #18589 Increment storage version to enable DICT_FSST in benchmark file by @@lnkuiper in #18588 [Fix] Hidden test failure in test_struct_update.test by @@taniabogatsch in #18598 correctly setting log transaction id in ThreadContext by @@xuke-hat in #18536 Backport renaming a config name small_block_size.json to block_size_16kB in NightlyTests by @@hmeriann in #18581 Update README.md by @@matthew-wright07 in #18614 [Test] Fix test case and a benchmark by @@hmeriann in #18610 [CI] Don't zip and upload Code Coverage tests results when Code Coverage got cancelled by @@hmeriann in #18607 [Profiling] Add client context into more read functions by @@maiadegraaf in #18514 bump httpfs by @@Tmonster in #18591 Fix serialization backwards compatability for varargs functions by @@Maxxen in #18596 Issue #18631: Streaming Windowed Quantile by @@hawkfish in #18636 parquet/parquet_multi_file_info.cpp: fix move from stack by @@carlopi in #18634 Adjust filter pushdown to latest polars release by @@pdet in #18624 Re-add hugeint to __internal_compress_string by @@jeewonhh in #18622 Add Field IDS to multi file reader for positional deletes by @@Tmonster in #18617 [CSV Sniffer] Fix type detection issue with union and empty columns by @@pdet in #18606 [ART] ART::Erase refactoring by @@taniabogatsch in #18595 wrap httplib ::max() call in WIN_32 check by @@Tmonster in #18590 Add enable verification config run by @@taniabogatsch in #18467 feat: add ETA to progress bar in DuckDB CLI by @@rustyconover in #18575 Add "Hash Zero" verification CI run by @@lnkuiper in #18623 Make more configs into generic settings by @@jeewonhh in #18592 bump avro to v1.4 by @@Tmonster in #18434 bump spatial (on main) by @@Maxxen in #18197 Change arrow() to export record batch reader by @@pdet in #18642 [Fix] Prevent logger deadlock by @@taniabogatsch in #18637 Remove PRAGMA enable_verification by @@taniabogatsch in #18645 Add 1.4 release codename by @@hannes in #18652 Python test runner: Fix result check for COPY ... RETURN_STATS queries by @@Flogex in #18625 Merge ossivalis into main by @@taniabogatsch in #18644 CLI: Make ETA more of an estimate, and support large_row_rendering for footers by @@Mytherin in #18656 Remove more PRAGMA enable_verification by @@taniabogatsch in #18664 [CI] skip building encodings extension in InvokeCI by @@hmeriann in #18655 Python test runner: Fix hash comparison error output by @@Flogex in #18626 [Dev] Add script to create patch from changes in an extension repository by @@Tishj in #18620 Correctly set weights in reservoir sample when switch to slow sampling by @@xuke-hat in #18563 Internal #5366: Window Interrupt Arguments by @@hawkfish in #18651 Remove PRAGMA enable_verification in more tests by @@taniabogatsch in #18670 [ Python SQLLogic Tester ] Add MERGE_INTO to statement.type enum in result.py by @@hmeriann in #18675 Load pandas in import cache before binding by @@evertlammerts in #18658 Internal #5662: IEJoin Test Plans by @@hawkfish in #18680 Correctly allocate uncompressed string data in ZSTD for many giant strings by @@Mytherin in #18678 Grab lock and double-check that column is not loaded in MoveToCollection by @@Mytherin in #18677 fix error message related to wrong memory unit by @@LiranBri in #18671 [CI] Temporarily skip triggering R Package Windows (Extensions) job by @@hmeriann in #18628 Fix the issue where delta_for isn't used in bitpacking when for is unavailable by @@xuke-hat in #18616 Add date_trunc() simplification rules by @@rcurtin in #18457 Expected errors 2053 by @@hmeriann in #14213 Internal #5366: Window State Arguments by @@hawkfish in #18676 Add WAL test config run by @@taniabogatsch in #18683 Using a different workflow to release the python package by @@evertlammerts in #18685 Make sure parse errors are wrapped in ErrorData by @@evertlammerts in #18682 [Python SQLLogicTest] Add test/sql/pragma/profiling/test_profiling_all.test to the SKIPPED_TESTS set by @@hmeriann in #18689 Issue #18457: DateTrunc Simplification Warnings by @@hawkfish in #18687 Expected errors 2053 by @@hmeriann in #18695 Hold row group lock for entire call of MoveToCollection by @@Mytherin in #18694 Unplug python (in ossivalis) by @@evertlammerts in #18699 Correctly handle collations for IN (subquery) by @@Mytherin in #18698 Move attached databases from a CatalogSet to a dedicated map of shared pointers by @@Mytherin in #18693 Make ART construction iterative via ARTBuilder by @@taniabogatsch in #18702 [Fix] Correctly handle table and index chunks in WAL replay buffering by @@taniabogatsch in #18700 Python-style positional/named arguments for macro's by @@lnkuiper in #18684 Internal #3273: Hashed Sort States by @@hawkfish in #18690 Add Option to Allocate Using an Arena in string_t by @@maiadegraaf in #17992 Fix issue with materialized CTE optimization in flatten_dependent_join by @@kryonix in #18714 [Profiling] Add Profiling to Read Function by @@maiadegraaf in #18661 Correctly throw an error when too few columns are supplied in MERGE INTO INSERT by @@Mytherin in #18715 Improved grammar generation script by @@NiclasHaderer in #18716 #Fix 18558: add row_group scan fast path by @@flashmouse in #18686 Added support for blob<->uuid conversions by @@dioptre in #18027 Minor fixes for other catalogs - mostly checking IsDuckTable() for unsupported operations by @@Mytherin in #18720 Fix PIVOT in multiple statements by @@evertlammerts in #18729 Internal #5669: Loop Join Thresholds by @@hawkfish in #18733 feat: enhance .tables command with schema disambiguation and filtering by @@shivampr in #18641 Add (CSV) file logger by @@samansmink in #17692 Use 1-based indexing for SQL-based JSON array extraction by @@lnkuiper in #18735 [unittest] SkipLoggingSameError() to make unittester report one failure per case by @@hmeriann in #18270 fix timetravel for default tables by @@samansmink in #18240 [C API] Function to set a copy callback for bind data by @@taniabogatsch in #18739 Secrets: if serialization_type is not specified, assume it's a key value secret by @@Mytherin in #18743 Merge ossivalis into main by @@taniabogatsch in #18719 Use correct type for pushing collations in subqueries by @@Mytherin in #18744 Add OS X notarization for DuckDB CLI and libduckdb.dylib by @@hannes in #18747 Add missing expected errors to the test cases by @@hmeriann in #18746 removed placeholder client directories for node and jdbc, its been > 1 yr by @@hannes in #18757 Append using a SQL query, instead of directly appending to a base table, and support user-provided queries through the QueryAppender by @@Mytherin in #18738 Backport #18374 to v1.3-ossivalis by @@lnkuiper in #18752 Add leak suppressions to nightly runs by @@Mytherin in #18748 Remove separate WAL encryption flag by @@Mytherin in #18750 Fixing lazy polars execution on query result by @@pdet in #18749 [Profiling] Add Profiling to Write Function by @@maiadegraaf in #18724 Extensions.yml should also check converted_to_draft by @@carlopi in #18754 Minor logging fixes and more benchmarking by @@samansmink in #18755 Add missing expected errors to the test cases (next chunk) by @@hmeriann in #18753 Refactor read_blob and read_text to use MultiFileFunction. by @@xevix in #18706 Add support for auto-globbing within a directory: if no matches are found for a specific path, we retry with /**/*.[ext] appended by @@Mytherin in #18760 Fix radix partitioning with more than 10 bits by @@ctsk in #18761 Fix index resolution when querying table with index via view by @@mach-kernel in #18319 Fix Path Typo in Extension's CMake Warning Message by @@beryllw in #18766 Make duckdb_log return a TIMESTAMP_TZ by @@Mytherin in #18768 Revert "Use 1-based indexing for SQL-based JSON array extraction" by @@Mytherin in #18758 [CI] Adjust test configs post logger PR by @@taniabogatsch in #18771 [Test Fix] Forward output to file by @@taniabogatsch in #18772 Propagate DUCKDB_*_VERSION in extensions and tests by @@Y-- in #18774 Add file_size_bytes (de-)serialization by @@lnkuiper in #18775 Use microsecond resolution for printing the current timestamp by @@Mytherin in #18776 Improve error messages for merge / vector reference by @@Mytherin in #18777 Move row id logic to separate RowIdColumnData class instead of inlining it into the RowGroup by @@Mytherin in #18780 Treat ENABLE_EXTENSION_AUTOINSTALL as the BOOL that it is by @@evertlammerts in #18778 Add memory_limit parameter to benchmark_runner/test_runner.py by @@lnkuiper in #18790 fix: improve speed of GetValue() for STRUCT type by @@rustyconover in #18785 Internal #3273: Parallel Window Masks by @@hawkfish in #18731 Task Scheduler: track exact task count, and re-signal on dequeue failure if there are tasks left by @@Mytherin in #18792 fix: coalesce query progress updates to reduce terminal writes by @@rustyconover in #18672 Support expressions as COPY file target by @@Mytherin in #18795 Remove everything python-package related by @@evertlammerts in #18789 Improve autocomplete suggestions by @@Dtenwolde in #18773 bump httpfs so it includes curl option by @@Tmonster in #18691 Issue #18767: Ignore Timestamp Offsets by @@hawkfish in #18794 Fixup progress_bar: avoid converting doubles into int32_t unchecked by @@carlopi in #18800 [chore] Fixup tidy-check on src/logging/log_manager.cpp by passing const & by @@carlopi in #18801 Internal #3273: Hashed Sort Callbacks by @@hawkfish in #18796 Typed macro parameters by @@lnkuiper in #18786 Fix some unindented interactions between EMPTY_RESULT_PULLUP and MATERIALIZED CTEs by @@kryonix in #18805 Add support for non-aggregate window functions by @@Maxxen in #18788 Expected errors 2053 by @@hmeriann in #18810 Test runner: Expand '{UUID}' into a random UUID by @@carlopi in #18809 Provide failing file name in Parquet reader error messages by @@Mytherin in #18814 [CI] install libcurl4-openssl-dev with apt-get by @@hmeriann in #18811 fix: Add COLLATE NOCASE support to strpos function by @@shivampr in #18819 Add callback to get a list of copy options, use this to provide suggestions and to erase options from import that are only used during exporting by @@Mytherin in #18812 For BC reasons - keep VARINT as alias for BIGNUM by @@Mytherin in #18821 [Fix] Bug in fixed-size buffer when throwing out-of-memory by @@taniabogatsch in https://github.com/duckdb/duckdb/pull/18769 Re-add accidentally removed check if copy_from is supported by @@Mytherin in https://github.com/duckdb/duckdb/pull/18824 Fix format-fix runs on Linux by @@staticlibs in https://github.com/duckdb/duckdb/pull/18827 Extensions.yml: Pass down save_cache to inner workflows by @@carlopi in https://github.com/duckdb/duckdb/pull/18828 Fix: Preserve database configuration flags for tab completion in DuckDB shell by @@rustyconover in https://github.com/duckdb/duckdb/pull/18482 Ensure a WAL file matches the DB file and checkpoint iteration by @@taniabogatsch in https://github.com/duckdb/duckdb/pull/18823 fix: sanitize input for enable_logging by @@samansmink in https://github.com/duckdb/duckdb/pull/18830 fix: silence warnings about signed/unsigned conversions. by @@rustyconover in https://github.com/duckdb/duckdb/pull/18835 Avoid expensive checkpoints and write amplification by appending row groups, and limiting vacuum operations for the last number of row groups by @@Mytherin in https://github.com/duckdb/duckdb/pull/18829 Fix/run function in transaction by @@Evannnnnnnn in https://github.com/duckdb/duckdb/pull/18741 add appender to concurrent test by @@c-herrewijn in https://github.com/duckdb/duckdb/pull/18721 Add support for reading/writing native parquet geometry types by @@Maxxen in https://github.com/duckdb/duckdb/pull/18832 Don't notify Py pkg when override git describe is set by @@evertlammerts in https://github.com/duckdb/duckdb/pull/18843 Avoid printing '99 hours', given in most cases that means estimate is… by @@carlopi in https://github.com/duckdb/duckdb/pull/18839 Add the VARIANT LogicalType by @@Tishj in https://github.com/duckdb/duckdb/pull/18609 Document storage version flag in CLI + minor rendering fix by @@Mytherin in https://github.com/duckdb/duckdb/pull/18841 Ignore null verification for statistics on structs by @@d-justen in https://github.com/duckdb/duckdb/pull/18813 Add OnBeginExtensionLoad callback by @@Mytherin in https://github.com/duckdb/duckdb/pull/18842 Bump MySQL/Postgres/SQLite by @@Mytherin in https://github.com/duckdb/duckdb/pull/18848 Merge ossivalis into main by @@taniabogatsch in https://github.com/duckdb/duckdb/pull/18844 Add test_env to unit tester by @@pdet in https://github.com/duckdb/duckdb/pull/18847 WAL <> DB File Match Fixes by @@Mytherin in https://github.com/duckdb/duckdb/pull/18849 Make ATTACH OR REPLACE atomic, keep list of used databases in MetaTransaction by @@Mytherin in https://github.com/duckdb/duckdb/pull/18850 Fix NULL path for json_each/json_tree by @@lnkuiper in https://github.com/duckdb/duckdb/pull/18852 No more wal_encryption flag by @@jeewonhh in https://github.com/duckdb/duckdb/pull/18851 Bump Ducklake by @@pdet in https://github.com/duckdb/duckdb/pull/18825 Add more encryption modes CTR and CBC by @@hannes in https://github.com/duckdb/duckdb/pull/18619 Centralize attached database paths in a DatabaseFilePathManager which is shared across databases created through the same DBInstanceCache by @@Mytherin in https://github.com/duckdb/duckdb/pull/18857 Hold segment lock during GetColumnSegmentInfo by @@Mytherin in https://github.com/duckdb/duckdb/pull/18859 update duckdb azure extension ref for 1.4.0 by @@benfleis in https://github.com/duckdb/duckdb/pull/18868 Expected errors 2053 by @@hmeriann in https://github.com/duckdb/duckdb/pull/18864 Add a FORCE_DEBUG flag to force -DDEBUG, similar to FORCE_ASSERT by @@Mytherin in https://github.com/duckdb/duckdb/pull/18872 Bump & remove patches for delta, avro, excel, encodings, fts by @@samansmink in https://github.com/duckdb/duckdb/pull/18869 [minor] Incompatible DB error message: add newline by @@carlopi in https://github.com/duckdb/duckdb/pull/18861 Bump mbedtls to v3.6.4 by @@Mytherin in https://github.com/duckdb/duckdb/pull/18871 Storage fuzzing + several fixes by @@Mytherin in https://github.com/duckdb/duckdb/pull/18876 Update ducdkb iceberg hash by @@Tmonster in https://github.com/duckdb/duckdb/pull/18873 [Test] Small fixes to concurrent attach/detach test by @@taniabogatsch in https://github.com/duckdb/duckdb/pull/18862 Internal #5796: Window Progress by @@hawkfish in https://github.com/duckdb/duckdb/pull/18860 Add COPY (...) TO ... (FORMAT BLOB) by @@Maxxen in https://github.com/duckdb/duckdb/pull/18840 Update spatial+vss+sqlsmith in preparation for v1.4 by @@Maxxen in https://github.com/duckdb/duckdb/pull/18882 Avoid automatically checkpointing if the database instance has been invalidated by @@Mytherin in https://github.com/duckdb/duckdb/pull/18881 Add COPY (FORMAT BLOB) to Andium too :^) by @@Maxxen in https://github.com/duckdb/duckdb/pull/18884 [C API] Result schema of prepared statements by @@hrl20 in https://github.com/duckdb/duckdb/pull/18779 Json: no reinterpret by @@carlopi in https://github.com/duckdb/duckdb/pull/18886 [Dev] Fix footgun in string_t::SetSizeAndFinalize by @@Tishj in https://github.com/duckdb/duckdb/pull/18885 [chore] Bump config test/configs/compressed_in_memory.json to new format by @@carlopi in https://github.com/duckdb/duckdb/pull/18888 bump aws and iceberg by @@Tmonster in https://github.com/duckdb/duckdb/pull/18889 Add rowsort to upsert_default.test by @@jeewonhh in https://github.com/duckdb/duckdb/pull/18890 fixing auto-specifying ciphers and remove double storage by @@hannes in https://github.com/duckdb/duckdb/pull/18891 Expected errors 2053 by @@hmeriann in https://github.com/duckdb/duckdb/pull/18892 Keep base data scan state alive in ColumnData::Update call by @@Mytherin in https://github.com/duckdb/duckdb/pull/18893 Add callback for when an extension fails to load, and also log this by @@Mytherin in https://github.com/duckdb/duckdb/pull/18894 Encryption now encoded as a bit, centralizing in set/getter by @@carlopi in https://github.com/duckdb/duckdb/pull/18897 Bump httpfs to v1.4-andium branch by @@carlopi in https://github.com/duckdb/duckdb/pull/18898 fix: refine query ETA display and Kalman filter stability by @@rustyconover in https://github.com/duckdb/duckdb/pull/18880 Bump inet & aws by @@samansmink in https://github.com/duckdb/duckdb/pull/18899 [chore] Fix amalgamation build in progress_bar by @@carlopi in https://github.com/duckdb/duckdb/pull/18910 Cannot create table from variant yet by @@Mytherin in https://github.com/duckdb/duckdb/pull/18912 In VerifyZeroReaders, get the header size from the buffer we are replacing instead of from the block manager by @@Mytherin in https://github.com/duckdb/duckdb/pull/18909 Fix #18152: avoid auto-detecting hive partitioning with COPY .. FROM by @@Mytherin in https://github.com/duckdb/duckdb/pull/18911 CLI: Correctly move to start of line by @@Mytherin in https://github.com/duckdb/duckdb/pull/18920 Strip question mark parameters from default temporary directory by @@Mytherin in https://github.com/duckdb/duckdb/pull/18915 Move Hash Zero CI run to nightly by @@Mytherin in https://github.com/duckdb/duckdb/pull/18925 Bump Iceberg by @@Tmonster in https://github.com/duckdb/duckdb/pull/18917 Issue template: Add the Python repository by @@szarnyasg in https://github.com/duckdb/duckdb/pull/18928 fix extension size increase by @@samansmink in https://github.com/duckdb/duckdb/pull/18923 [Dev] Fix reference of uninitialized memory in Variant conversion first pass by @@Tishj in https://github.com/duckdb/duckdb/pull/18921 Bump DuckLake to Latest Main by @@pdet in https://github.com/duckdb/duckdb/pull/18926 Make row-group metadata re-use experimental for now by @@Mytherin in https://github.com/duckdb/duckdb/pull/18922 Fix exception propagation in C API by @@mlafeldt in https://github.com/duckdb/duckdb/pull/18924 Bump httpfs by @@carlopi in https://github.com/duckdb/duckdb/pull/18930 Bump ducklake and don't write empty bbox in geoparquet stats by @@Maxxen in https://github.com/duckdb/duckdb/pull/18936 [PROFILING] Fix EXPLAIN ANALYZE returning empty results when PRAGMA enabled_profiling = 'no_output' by @@maiadegraaf in https://github.com/duckdb/duckdb/pull/18935 Http_util can return success for all [200, 300) responses, as well as redirects by @@Tmonster in https://github.com/duckdb/duckdb/pull/18940 Fix TransformStringToLogicalType for enums arrays by @@tdoehmen in https://github.com/duckdb/duckdb/pull/18941 [unittester] Allow overriding data/ folder to custom location by @@carlopi in https://github.com/duckdb/duckdb/pull/18929 Unpin fixed-size sorting keys by @@lnkuiper in https://github.com/duckdb/duckdb/pull/18945 Add missing parameters to COPY ... (FORMAT JSON) by @@lnkuiper in https://github.com/duckdb/duckdb/pull/18946 Fixes for encrypted database, make cross-engine encryption work, and expand testing by @@hannes in https://github.com/duckdb/duckdb/pull/18951 fix windows linking issue ducklake by @@samansmink in https://github.com/duckdb/duckdb/pull/18953 bump iceberg by @@Tmonster in https://github.com/duckdb/duckdb/pull/18957 [SQLLogicTest] Detect errors thrown in LoadExtension of the require statement by @@Tishj in https://github.com/duckdb/duckdb/pull/18950 Don't use VectorOperations::Copy for string dictionary hashes by @@lnkuiper in https://github.com/duckdb/duckdb/pull/18949 Fix error reporting in SSLClient by @@staticlibs in https://github.com/duckdb/duckdb/pull/18958 bump spatial by @@Maxxen in https://github.com/duckdb/duckdb/pull/18961 Allow extensions to customize ATTACH OR REPLACE conflict behavior by @@ywelsch in https://github.com/duckdb/duckdb/pull/18962 Unify test runner keyword replacement, and don't run LOAD [ext] by default by @@Mytherin in https://github.com/duckdb/duckdb/pull/18963 [chore] Bump httpfs and remove patches by @@carlopi in https://github.com/duckdb/duckdb/pull/18965 Correctly update row group data pointers and root table pointer after checkpoint by @@Mytherin in https://github.com/duckdb/duckdb/pull/18966 Attach: Cleanup duplicate data path handling, and make IF NOT EXISTS no longer abort if we are adding a path with the same name by @@Mytherin in https://github.com/duckdb/duckdb/pull/18974 Bump DuckLake and HTTPFS by @@pdet in https://github.com/duckdb/duckdb/pull/18975 Issue #18971: Empty Unsorted Windows by @@hawkfish in https://github.com/duckdb/duckdb/pull/18976 Check context.interrupted flag in table scan by @@Mytherin in https://github.com/duckdb/duckdb/pull/18981 Only return cgroup memory limit if it's a sane value by @@szarnyasg in https://github.com/duckdb/duckdb/pull/18668 Macro fixes by @@lnkuiper in https://github.com/duckdb/duckdb/pull/18992 ATTACH IF NOT EXISTS - wait until database is fully attached before returning by @@Mytherin in https://github.com/duckdb/duckdb/pull/18993 WALReplay Fix: In UpdateColumn, no longer assume all updates are part of the same vector, but instead verify this and batch updates per vector by @@Mytherin in https://github.com/duckdb/duckdb/pull/18999 Bump iceberg by @@Tmonster in https://github.com/duckdb/duckdb/pull/19001 @ text @# $NetBSD: Makefile,v 1.11 2026/01/07 08:46:08 wiz Exp $ DISTNAME= duckdb-1.4.4 CATEGORIES= databases MASTER_SITES= ${MASTER_SITE_GITHUB:=duckdb/} GITHUB_PROJECT= duckdb GITHUB_TAG= v${PKGVERSION_NOREV} MAINTAINER= ryoon@@NetBSD.org HOMEPAGE= https://duckdb.org/ COMMENT= In-process SQL OLAP database management system LICENSE= mit USE_LANGUAGES= c c++ USE_TOOLS+= gmake # for alloca(3) FORCE_CXX_STD+= gnu++11 CMAKE_CONFIGURE_ARGS+= -DBUILD_EXTENSIONS='autocomplete;core_functions;icu;jemalloc;json;parquet;tpcds;tpch' CMAKE_CONFIGURE_ARGS+= -DBUILD_BENCHMARKS=ON CMAKE_CONFIGURE_ARGS+= -DBUILD_TPCE=ON CMAKE_CONFIGURE_ARGS+= -DWITH_INTERNAL_ICU=True CMAKE_CONFIGURE_ARGS+= -DDUCKDB_EXPLICIT_VERSION=${PKGVERSION_NOREV} .include "../../devel/cmake/build.mk" .include "../../security/openssl/buildlink3.mk" #.include "../../textproc/icu/buildlink3.mk" .include "../../mk/bsd.pkg.mk" @ 1.11 log @*: recursive bump for icu 78.1 @ text @d1 1 a1 1 # $NetBSD: Makefile,v 1.10 2025/06/15 13:30:19 ryoon Exp $ d3 5 a7 2 PKGREVISION= 1 .include "../../databases/duckdb/Makefile.common" d28 1 a28 1 .include "../../textproc/icu/buildlink3.mk" @ 1.10 log @databases/duckdb: Update to 1.3.0 * Prepare for databases/py-duckdb. Changelog: 1.3.0: What's Changed V1.2 histrionicus by @@Mytherin in #16070 V1.2 histrionicus by @@Mytherin in #16072 unittests: clear test directory after every test by @@Mytherin in #16053 Benchmark runner: catch and log errors + add support for retry load N syntax by @@Mytherin in #16054 Throw an error when unsupported commands are used in concurrentloop by @@Mytherin in #16009 Remove extension definitions to prevent re-compilation of the entire system on commit by @@Mytherin in #15955 Display schema information of currently selected database only by @@ashwaniYDV in #15815 Issue #14366: Average Intervals by @@hawkfish in #15864 Internal #2176: Temporal AVG by @@hawkfish in #15661 discussions #15981: remove confusing comment in "duckdb/tools/shell/shell.cpp" by @@komainu8 in #15984 Fix #15466 Transform LIMIT or OFFSET first based on order specified in prepared statement by @@ashwaniYDV in #15484 Bitpacking mode info by @@arjenpdevries in #15623 Sniff Timestamp_TZ from CSV FIles by @@pdet in #15730 [no-op] Add documentation for filesystem read behavior by @@dentiny in #15937 Accept "Auto" as date/timestamp format by @@pdet in #15808 Parquet Reader Cleanup: Move ColumnReaders to separate files by @@Mytherin in #16092 Parquet Reader: Move decoding logic into separate Decoder classes by @@Mytherin in #16100 BundleStaticLibs to be also triggered by InvokeCI by @@carlopi in #16107 Parquet Reader: Split DeltaLengthByteArray decoder from DeltaByteArray, and read the strings in a streaming manner by @@Mytherin in #16105 Parquet Dictionary reader: set NULL values as the last value in the dictionary by @@Mytherin in #16106 Parquet Reader: Share ResizeableBuffers across decoders, and unify Plain/PlainReference by @@Mytherin in #16113 Using GitHub ARM runners for Linux CLI builds by @@hannes in #16119 Parquet Reader: Implement dedicated Skip method by @@Mytherin in #16117 Use ColumnSegment::FilterSelection and SelectionVector for filtering in Parquet scans by @@Mytherin in #16126 [Dev] Fix output (long lines > 333 characters) getting truncated in shell by @@Tishj in #16128 Adaptive table filter: initialize filter order based on heuristics by @@Mytherin in #16127 Feature #16044: TimeZone Offset Seconds by @@hawkfish in #16048 ATTACH OR REPLACE database to allow swapping of new data. by @@xevix in #15355 [Dev] Remove upsert_conflict_in_different_chunk.test by @@Tishj in #15980 [Dev] Fix issue related to unpacked columns and the NOT operator by @@Tishj in #15534 [Julia] Add support for named params in prepared statements by @@tqml in #15621 Use Adaptive Filters in the Parquet reader by @@Mytherin in #16133 Parquet reader: push table filters directly into dictionaries by @@Mytherin in #16136 Parquet reader: Plain templates - make CHECKED a template parameter, and use memcpy/bulk skip when reading/skipping without defines by @@Mytherin in #16141 Parquet reader: only set invalid entry in the dictionary when the column has defines by @@Mytherin in #16144 Add uniq_ptr_cast for interpreted benchmark. by @@Tmonster in #16151 Hopefully fixing ci runs by @@hannes in #16150 Removed the last CI job that used the Ubuntu 18 setup by @@hannes in #16155 Parquet Reader: Split CreateReader into two separate stages - ParseSchema and CreateReader by @@Mytherin in #16161 Have CSV Parellel tests on CI again by @@pdet in #16164 [Python][Dev] Bump the minimum pybind11 version from 2.6 to 2.9 by @@Tishj in #16159 Add StackTraces to FatalExceptions by @@NiclasHaderer in #16158 Rework invoke by @@carlopi in #16108 Adds pre-optimization hooks for DuckDB by @@NiclasHaderer in #16115 Unify behavior of range/generate_series with PostgreSQL by @@kryonix in #15935 [CI] Avoid Linux CLI jobs to fail-fast by @@carlopi in #16173 Parquet: Add dedicated Select method that can be used to push selection vectors into the read by @@Mytherin in #16174 Unvendor ICU by @@m-kuhn in #16176 Parquet reader: batch check if buffer is available in RLEBpDecoder by @@Mytherin in #16185 Parquet Reader: for DeltaLengthByteArray encoding, directly refer to strings from the block without copying by @@Mytherin in #16186 unified names for duckdb-extensions by @@hmeriann in #16179 Only delete test directory when --test-temp-dir is not specified by @@Mytherin in #16192 Fix #16163: COLUMNS should not treat identifiers as strings by @@Mytherin in #16193 Parquet reader: Avoid applying bloom filters if we are casting columns by @@Mytherin in #16194 Pretty print sniffer values by @@pdet in #16182 V1.2 histrionicus by @@Mytherin in #16191 Bump Julia by @@Mytherin in #16199 Ensure that dependent targets are present after find_package. by @@BillyONeal in #16197 Concurrency groups for R and Wasm by @@hmeriann in #16201 Parquet Writer Cleanup: Move ColumnWriters to separate files by @@Mytherin in #16202 [fix] Use bigobj when building with MSVC by @@m-kuhn in #16200 Improve performance of UNNEST/UNPIVOT by using selection vectors to unnest multiple lists at once by @@Mytherin in #16210 Add the TRY expression by @@Tishj in #15939 [Python][Dev] Replace the default connection when it's closed by @@Tishj in #16160 Use steady clock for profiler by @@dentiny in #16198 Add parallel memset ting the input columns of the UNPIVOT operator in the UNNEST by @@Mytherin in #16221 Left join push down optimization by @@Damon07 in #15881 Do In-Filter pushdown in PyArrow by @@pdet in #16224 Use _win32 with MSVC by @@cfis in #16235 Fix PythSV/JSON readers by @@Mytherin in #16248 Modify histogram test to more fuzzily check boundaries since the test can be inconsistent on different platforms by @@Mytherin in #16261 [Dev] Fix issue in TRY expression with dictionary_expression Vector veri Fix extension install mode null by @@samansmink in #16268 A little cleanup. by @@JasonPunyon in #16028 Improve Parquet writer performance by @@lnkuiper in #16243 Merge v1.2-histrionicus into main by @@Mytherin in #16284 Many reclaim storage6246 Add pragma to truncate duckdb log storage by @@samansmink in #16274 Some more Parquet writer performance improvements by @@lnkuiper in #16287 Do duckdb_extract_statements to be able to execute pivot in ADBC by @@pdet in #16162 [Dev] Improve/Add handling of escapes in VARCHAR -> list/struct/map and align behavior by @@Tishj in #15944 make ValidityMask::RowIsValidUnsafe really unsafe by @@xuke-hat in #16302 Multi File Reader Rework: Add MultiFileReaderFunction that is used to wrap a single-file reader, and use it for the Parquet reader by @@Mytherin in #16299 [Python Dev] Add support for fully qualified references in .table(...) method by @@Tishj in #16291 [Dev] MultiFileReader - Add to the column_indexes for file_row_number by @@Tishj in #16311 Parquet reader performance by @@lnkuiper in #16315 Bump Julia FixedPointDecimals dependency version by @@mbarbar in #16323 Merge V1.2 histrionicus into main by @@Mytherin in #16324 Add new recursive semantics (USING KEY) by @@cryoEncryp in #12430 fix: add StringStats::SetMaxStringLength by @@rustyconover in #16326 Fix decorrelation of WITH USING KEY by @@kryonix in #16330 Issue #16250: Window Range Performance by @@hawkfish in #16320 Verify UTF-8 in DeltaLengthByteArrayDecoder and speed it up by @@lnkuiper in #16328 Add missing include by @@Mytherin in #16342 [chore] No ccache for OSX Python by @@carlopi in #16348 Linux CLI: override platform for ARM manylinux by @@carlopi in #16347 docs: tweak explanation of median for even cardinality inputs by @@NickCrews in #13726 [parquet] Fix implicit idx_t to int64_t conversion flagged by clang-tidy by @@carlopi in #16368 Improve error message on failure to install local extension by @@carlopi in #16371 MAIN_BRANCH_VERSIONING: main branch to get descriptors like v1.3.0-dev1234 instead of v1.2.1-dev1234 by @@carlopi in #16366 Parallel HT Zeroing: Set entries_per_task so that there are 4x more tasks than threads by @@gropaul in #16301 Internal #2176: SUMMARIZE Temporal Types by @@hawkfish in #16095 [SwiftRelease CI] fetch tags before checking there is already a tag with the same name by @@hmeriann in #16376 Push Select into ArrayColumnData to avoid scanning arrays that are not required for the query by @@Mytherin in #16356 Revert "Linux CLI: override platform for ARM manylinux" by @@carlopi in #16374 Rework CSV Reader: use the new MultiFileReaderFunction interface by @@Mytherin in #16349 Add connection and transaction identifiers by @@samansmink in #16296 Add parquet 'unknown' logical type by @@hannes in #16378 Internal #4287: INTERVAL Times DOUBLE by @@hawkfish in #16386 pb/compressed vector serialization by @@peterboncz in #16066 Fix issue #16377 by @@kryonix in #16391 Read support for Parquet Float16 by @@hannes in #16395 MAIN_BRANCH_VERSIONING: Adopt also for Python build and amalgamation by @@carlopi in #16400 Fuzzer Fix: Fix Avg for NULL cast to TIMESTAMP by @@Tmonster in #16394 [FriendlySQL] Expand functionality of the Unpacked COLUMNS expression by @@Tishj in #16290 Python Client: Faster Python Object Conversion by @@Mytherin in #16431 Fixup #16400 by correctly passing down SETUPTOOLS_SCM_PRETEND_VERSION by @@carlopi in #16435 Issue #16250: Window Range Performance by @@hawkfish in #16438 Merge v1.2-histrionicus into main by @@Mytherin in #16439 MAIN_BRANCH_VERSIONING: Add also prefix_version by @@carlopi in #16441 [no-op] Remove unused function GetValueRefUnsafe by @@dentiny in #16440 Filter Combiner Clean-up: move filter pushdown to separate functions, remove old commented out code by @@Mytherin in #16443 [Python] Add the SQLExpression method to the Expression API by @@Tishj in #16424 [Dev] Mention the problematic type in UNNEST BinderException by @@Tishj in #16429 Merge v1.2 into main again by @@Mytherin in #16447 Filter Combiner: Allow rowid pushdown for IN/OR filters and pushdown for temporal types by @@Mytherin in #16450 Parquet: always launch max threads if we are scanning multiple files by @@Mytherin in #16457 fix documents of C functions by @@yiyuanliu in #16357 Add a TableFilterState for execution of table filters by @@Mytherin in #16461 Mirror discussions to the internal repository by @@szarnyasg in #16464 Rework JSON Reader: use the new MultiFileReaderFunction interface by @@Mytherin in #16477 Speed-up contains by using memchr on every iteration by @@Mytherin in #16484 Fix error cases by @@Y-- in #16494 Prevent external joins (if possible) by @@lnkuiper in #16430 Merge v1.2 into main by @@Mytherin in #16517 Optimize FSST decoding by @@lnkuiper in #16508 Extract subsystem by name by @@dentiny in #16226 Avoid throwing an exception (that is then swallowed) when computing compressed materialization over stats that are not set by @@Mytherin in #16532 Checksum backward compatibility by @@lnkuiper in #16505 Prefetch Parquet page header by @@lnkuiper in #16507 Let GitHub render *.test files as SQL by @@mlafeldt in #16534 Fix ADBC to properly quote table and schema names by @@CurtHagenlocher in #16526 Pass ClientContext to catalog initialize, and postpone index binding when replaying the WAL by @@Mytherin in #16536 Allow UNITTEST_ROOT_DIRECTORY to be configured through CMake by @@Mytherin in #16540 Internal #4347: ISO Year Week by @@hawkfish in #16567 throw() -> noexcept in skiplist by @@r-barnes in #16548 Fix test/sql/aggregate/aggregates/histogram_table_function.test to pass the Linux CLI (arm64) CI by @@hmeriann in #16538 feat: move GRANT from reserved to unreserved keyword by @@stephaniewang526 in #16546 Python test runner: Avoid enabling profiling when executing restart command by @@Flogex in #16547 Add duckdb_prepared_statements by @@Tishj in #16541 [minor] Keep bit type sanity check consistent by @@dentiny in #16575 Support CREATE TABLE AS ... WITH NO DATA by @@hannes in #16586 Parquet FLOAT16 - fix cast by @@hannes in #16580 remove invalid tokens from nanosecond example by @@hamilton in #16577 CrossVersion.yml: Add v1.2.1, v1.2-histrionicus and main by @@carlopi in #16576 Fix #16524: DEPENDENT_JOIN may not flatten by @@flashmouse in #16537 [Julia] Add support for appending duckdb List types by @@era127 in #16512 [PySpark] - Add expr function by @@mariotaddeucci in #16468 regex_replace no longer swallows regex errors by @@hannes in #16380 Parquet Writer Clean-up: Split CreateWriterRecursive into two methods, and use ParquetColumnData for writer as well by @@Mytherin in #16592 Bump Julia to 1.2.1 by @@Mytherin in #16593 Improved appender error message by @@NiclasHaderer in #16599 Change static variables to be on the stack instead by @@Y-- in #16597 Add support for RETURN_STATS to COPY by @@Mytherin in #16595 Better error messages for the CSV Scanner by @@pdet in #16585 Support Enum types in read_csv - Python by @@pdet in #15710 Fix CI Tidy by @@pdet in #16610 Add some minor helper functions (QueryResultIterator::IsNull and casts to MultiFileList/Reader by @@Mytherin in #16611 Add support for ALTER TABLE tbl SET PARTITIONED BY (key1, key2, ...) in the grammar by @@Mytherin in #16612 Issue template: direct UI issues to the UI repository by @@szarnyasg in #16619 [Dev] Make the various mappings in MultiFileReaderData typesafe by @@Tishj in #16596 Bump mbedtls to 3.6.2 and re-apply patches by @@hannes in #16485 Read and Write Complex Json from Arrow Types by @@pdet in #16385 Add Docker support for RISC-V CI with appropriate build commands by @@mocusez in #16549 Fix missing **kwargs in adbc_driver_duckdb.dbapi.connect() by @@davlee1972 in #16637 [Dev] Clean up and fix the CGroup memory/cpu limit discovery logic by @@Tishj in #16608 Expose Value::ToSQLString() in C API by @@mt-caret in #16471 Add the missing binding for json_serialize_sql by @@liznear in #16666 Do not create validity mask for non-null const vector by @@xuke-hat in #16669 Fix #16665: fix parquet multi_reader bloom_probe logic error by @@flashmouse in #16677 Add alias to catalog by @@c-herrewijn in #16600 Decouple physical operator ownership from operators by @@taniabogatsch in #16545 cmake: fix external icu by @@autoantwort in #16676 Character length and date functions by @@hannes in #16653 [Dev] Don't try to include third_party/mbedtls/VERSION with package_build.py by @@Tishj in #16683 Add -ui to CLI help text by @@akx in #16626 Fix alias of column reference lost in ReplaceProjectionBindings by @@Damon07 in #16686 Merge v1.2-histrionicus into main by @@Mytherin in #16687 Fix for GCC-4.8 by @@Mytherin in #16690 JSON Reader: make read_position atomic so this can be read by the progress bar while processing the JSON file by @@Mytherin in #16692 [Julia] support binding for vectors by @@slwu89 in #16701 Make CSV Parser strict_mode=True fail on a mix of new line delimiters. by @@pdet in #15959 [pypi] Fix cleanup logic for multiple branches by @@hmeriann in #16634 Add support for ALTER TABLE tbl SET SORTED BY (key1 DESC, key2, ...) in the grammar by @@Mytherin in #16714 RETURN_STATS: remove footer_offset, and emit written partition keys by @@Mytherin in #16715 In case all rows of a CSV batch are errors, we continue processing by @@pdet in #16713 add workaround for patching httpfs ext by @@samansmink in #16722 Implement UUID v7 by @@dentiny in #15819 Fix roundtripping of stringified nested types by @@Tishj in #16304 Add Notify External Repositories Workflow by @@maiadegraaf in #16730 Expose a selection vector and the Slice method to the C API by @@joseph-isaacs in #16696 Add support for tracking column_size_bytes and contains_nan in RETURN_STATS by @@Mytherin in #16731 Add support for WRITE_EMPTY_FILE option to COPY - which allows skipping of writing empty files by @@Mytherin in #16737 Parquet Writer: Truncate string stats for large strings, instead of bailing on writing stats by @@Mytherin in #16736 RLE compression - memset alignment bytes to zero when aligning the counts by @@Mytherin in #16735 Write UUID stats to Parquet files and support reading uuid stats by @@Mytherin in #16744 Add an initial value to list_reduce by @@maiadegraaf in #16602 shell: make -bail work for more errors by @@mlafeldt in #16594 Call Notify External Repositories from Invoke CI by @@maiadegraaf in #16747 JSON bugfixes by @@lnkuiper in #16729 Add support for dynamically providing extra info post-execution in table functions, and use this to emit the total number of files read by the MultiFileReader by @@Mytherin in #16749 [Python Dev] Fix the versioning of the nightly python builds by @@Tishj in #16739 shell: fix sometimes-uninitialized error by @@mlafeldt in #16761 Issue #16250: Window Range Performance by @@hawkfish in #16765 Avoid building Python 3.7 wheels also for Linux by @@carlopi in #16769 Pyodide 0.27.2: conditionally skip tests by @@carlopi in #16772 Push catalog lookups through an extensible EntryLookupInfo struct by @@Mytherin in #16764 Fix two minor problems with NotifyExternalRepositories / odbc by @@carlopi in #16776 update expected results reflecting the changes brought ups with Fix roundtripping of stringified nested types PR by @@hmeriann in #16775 Merge V1.2 -> Main by @@pdet in #16751 Add support for time travel syntax in the FROM clause by @@Mytherin in #16774 Python docs: List all join types by @@szarnyasg in #16789 [chore] NotifyExternalRepositories.yml: Fix endpoint to be pinged by @@carlopi in #16793 Remove delta from extensions built on a nightly basis (vs main branch) by @@carlopi in #16795 OSX.yml & Windows.yml: remove repository_dispatch, already handled by InvokeCI by @@carlopi in #16796 Make extensions be linked privitally into duckdb by @@JAicewizard in #16726 Add additional iterations to avoid assertion failure in TemporaryMemoryManager by @@lnkuiper in #16801 Change the STANDARD_MASK_SIZE calculation to use size of template type. by @@sebastiaan-dev in #16807 Fix nightly table sample error by @@Tmonster in #16811 Fix tidy by @@pdet in #16805 support 'categories' label in function catalog by @@c-herrewijn in #15654 regenerate function headers by @@c-herrewijn in #16822 Internal #4490: Window Jump Reset by @@hawkfish in #16816 Regression.yml: Actually checkout proper base.sha commit by @@carlopi in #16824 fix: drop useless python import by @@yihong0618 in #16808 NightlyTests.yml: Inline env variables into build command by @@carlopi in #16817 Benchmark runner summary by @@hmeriann in #16759 Add storage_version 66 for version 1.3.0 by @@carlopi in #16800 Revert "fix: drop useless python import" by @@Mytherin in #16834 [MultiFileReader] Rework MultiFileReader::FinalizeChunk to use Expressions by @@Tishj in #16630 Merge v1.2 into main by @@Mytherin in #16832 Fix NULL key handling in mark join by @@xuke-hat in #16825 compressed vector serialization fixes by @@peterboncz in #16648 really sorry about this by @@peterboncz in #16840 Fix Python docstrings for unique by @@szarnyasg in #16845 [MultiFileReader] Create "local" filters to hand to underlying readers by @@Tishj in #16838 Revert "Regression.yml: Actually checkout proper base.sha commit" by @@Mytherin in #16860 [ART] Immediately erase empty fixed-size buffers by @@taniabogatsch in #16727 Resolve defaults and column index map by pushing a Projection (instead of executing in the insert itself) by @@Mytherin in #16867 Fix issue with sorting dev versions in pypi_cleanup.py script to keep on PyPi the most recent dev versions by @@hmeriann in #16873 Allow filters to be pushed through joins if there are projection maps by @@lnkuiper in #16871 Expressions in create secret by @@samansmink in #15801 Python - Arrow IPC support in from_arrow by @@pdet in #16821 [ART] Introduce a new ARTScanner and make InitMerge and Vacuum iterative by @@taniabogatsch in #16861 Do not pushdown filters which bindings only match the right side of the left join by @@Damon07 in #16880 MultiFileReader Rework (part 17) - remove MultiFileReaderData - and move as much as possible out of the file readers by @@Mytherin in #16882 ICU: Unify TimeZone accessing code by @@Mytherin in #16887 Rework ICU age computation to convert to a timestamp and use the regular interval age computation by @@Mytherin in #16889 Reduce allocations during aggregations by @@lnkuiper in #16849 CI: Prevent marking issues as 'stale' if they have the 'no stale' label by @@szarnyasg in #16903 Add field name to log line which fails Parquet spec by @@jsbali in #16862 Internal #4490: Window Threading Cleanup by @@hawkfish in #16879 Adding gzip version of shell for linux/osx install script by @@hannes in #16116 Fix USING KEY reference error by @@kryonix in #16906 [Nested] Enable Varargs in LIST_CONCAT by @@maiadegraaf in #16870 Fix several issues with vsize=2, and move vsize=2 tests to Main.yml by @@Mytherin in #16918 C API comments: Fix a/an typos by @@szarnyasg in #16925 Reduce locking with FILE_SIZE_BYTES/ROW_GROUPS_PER_FILE in Parquet writer by @@lnkuiper in #16928 [Python] Fix annotation of condition argument in join so it accepts Expression by @@MarcoGorelli in #16933 Fix GCC 4.8 and add it back to Main workflow by @@Mytherin in #16937 Merge v1.2 into main again by @@Mytherin in #16939 MultiFileReader - Perform nested remapping of field indexes instead of relying on casts by @@Mytherin in #16941 Internal #4552: Short Circuit CSE by @@hawkfish in #16931 Add back manylinux extensions by @@carlopi in #16944 Run CI on merge group by @@Mytherin in #16945 Internal #4516: Interval BIGINT Variants by @@hawkfish in #16904 Split query string for multi-statement queries by @@Mytherin in #16955 Vector Verification: Rework to run based on env variable DUCKDB_DEBUG_VERIFY_VECTOR and move to Main.yml by @@Mytherin in #16957 Move the no string inline/alternative verify workflow to Main.yml by @@Mytherin in #16958 [Python] Tighten type annotations on shape and columns by @@MarcoGorelli in #16948 Pass down CMAKE_POLICY_VERSION_MINIMUM and fix for local development by @@carlopi in #16953 [ART] Use the ARTScanner for VerifyAllocations (make it iterative) by @@taniabogatsch in #16946 Move ThreadSanitizer test from nightly test to Main, and fix locking issue by @@Mytherin in #16960 Re-enable workflows to run on PRs by @@Mytherin in #16961 Fix for selecting NaN values from Parquet files by @@Mytherin in #16962 Move LatestStorage tests to NightlyRelease - and fix issue with overflow string blocks not being cleaned up correctly by @@Mytherin in #16972 Arena-allocate physical operators by @@taniabogatsch in #16911 Make file_row_number a virtual column, and support per-file virtual columns in the MultiFileReader by @@Mytherin in #16979 Add a setting scheduler_process_partial that allows partial scheduling of tasks in the background threads by @@Mytherin in #16973 Clean up format script, gather all files then run concurrently instead of running concurrently per directory by @@Mytherin in #16988 Add support for altering struct columns (adding fields, dropping fields, renaming fields) by @@Mytherin in #17003 Fix CSV fuzzer tests by @@pdet in #16994 [Fix] Keep original expression for macro + lambda's with subqueries by @@taniabogatsch in #17020 Detect when tables have been dropped or altered, and prevent deletes in this scenario by @@Mytherin in #17018 Update links pointing to duckdb.org by @@szarnyasg in #16999 Fix for joining on floating columns #16901 by @@nickzoic in #16965 fix: remove ununsed stream struct member from ArrowScanLocalState by @@rustyconover in #17023 [Dev] Use UnifiedVectorFormat instead of a flattened Vector in UpdateSegment::Update by @@Tishj in #16974 Remove Arrow Extenson from core extensions by @@pdet in #17027 Correctly propagate ClientContext to TaskExecutor by @@ywelsch in #17026 Issue #17001: AsOf memory Management by @@hawkfish in #17028 [MultiFileReader] Make it possible for the multi file reader to add a DeleteFilter to the BaseFileReader by @@Tishj in #17032 Add optional OVERRIDE_NEW_DELETE build parameter by @@lnkuiper in #17035 Clean-up virtual columns and make MultiFileReader::InitializeReader virtual by @@Mytherin in #17038 Allow a table to define their own row-id columns for delete/update, instead of assuming it is always COLUMN_IDENTIFIER_ROW_ID by @@Mytherin in #17039 Handle Parquet with compressed empty DataPage v2 by @@EnricoMi in #17031 Combine small row groups in Parquet writer by @@lnkuiper in #17036 Merge v1.2.2 into main by @@carlopi in #17037 implement function so I can send a patch to httpfs by @@lnkuiper in #17048 FORCE_ASYNC_SINK_SOURCE: pass also to unittester by @@carlopi in #17053 If a Max Line Size Error happens on all CSV dialect candidates, throw a max line size error. by @@pdet in #16935 Expose BindExtraColumns as a public function by @@Mytherin in #17060 trigger .github/workflows/NightlyBuildsCheck.yml from external repo by @@hmeriann in #16949 Minor parquet crypto clean-up: allow footer key to be passed in directly, and avoid constantly re-reading the key from the config by @@Mytherin in #17070 update julia to v1.2.2 by @@Maxxen in #17074 MultiFileReader Rework (part 18): Replace file path with OpenFileInfo struct by @@Mytherin in #17071 Fix httpfs patches: avoid git log since might contain unsanitised error word by @@carlopi in #17075 Re-enable Avro on core by @@Tishj in #17072 [Nested] Optimize List Type in list_value by @@maiadegraaf in #17063 Grow string dictionary dynamically in Parquet writer by @@lnkuiper in #17061 Add extended file info to OpenFileInfo, and use this to pass encryption keys and footer size to Parquet reader by @@Mytherin in #17085 [Dev] Automatically re-execute when calling __arrow_c_stream__ on an already-consumed-result by @@Tishj in #17087 fsst: Avoid to propagate alignment information in FSST_UNALIGNED_STORE by @@carlopi in #17094 Fix sqlite3 api wrapper link + remove R-CMD-check + add more nightly tests by @@carlopi in #17095 support large dictionary value and constant vector creation in the C API by @@joseph-isaacs in #17064 Add missing lock to UpdateSegment::FetchRow, and cleanup API to require the lock by @@Mytherin in #17100 Valgrind requires tpch by @@carlopi in #17101 Switch to manylinux_2_28 by @@hannes in #16956 Changing mbedtls encryption API by @@ccfelius in #16196 Pull OpenFileExtended through the opener and virtual file system layers by @@Mytherin in #17102 Fix an issue in upserts where the local append state was not correctly flushed by @@Mytherin in #17109 Always parallelize read_json schema detection by @@lnkuiper in #17106 Move transaction cleanup outside of the transaction lock by @@taniabogatsch in #17034 Remove R_CMD_CHECK.yml, now handled by duckdb/duckdb-r repo by @@carlopi in #17127 JSON Bugfixes by @@lnkuiper in #17119 Refactor relassert runs, adding some variations in compiler / statically linked extensions by @@carlopi in #17104 extension-upload-from-nightly.sh: Add --region by @@carlopi in #17120 MultiFileReader: several fixes for virtual column handling and make virtual column handling extensible by @@Mytherin in #17123 Remove misleading lock comment in data table by @@taniabogatsch in #17125 [Dev] Add "registries" to vcpkg.json, add script to list the packages of the registry. by @@Tishj in #17124 External File Cache by @@lnkuiper in #16463 Notify nightly build status by @@hmeriann in #17108 Strict UUID cast by @@lnkuiper in #17138 Copy To File: avoid calling Combine for threads that have not written any rows by @@Mytherin in #17142 Add file_index virtual column to the multi file reader that returns the file index of the read file by @@Mytherin in #17144 MultiFileReader: simplify constant handling, and allow virtual columns returned by the multi file reader to be constant by @@Mytherin in #17149 Changes to encodings to make them more flexible to replacement maps. by @@pdet in #17146 Optimize large Top N queries by @@lnkuiper in #17141 Only trigger TopN rewrite relatively small limits compared to the table size. by @@Tmonster in #17140 platform.hpp: Propagate DUCKDB_EXPLICIT_PLATFORM, avoid early return by @@carlopi in #17137 Keeping the filters which do not remove NULL values by @@Damon07 in #17045 Improve FileSync call on unix platform by @@dentiny in #16893 README: Fix to building link by @@szarnyasg in #17161 [InvokeCI] Add missing pipe to run instruction by @@hmeriann in #17163 Internal #4667: 2025b TimeZone Data by @@hawkfish in #17160 Unify function list by @@c-herrewijn in #17168 [Dev] Generate the EXTENSION_SECRET_TYPES instead of hardcoding them by @@Tishj in #17183 Fix grouping feature with interval type by @@handstuyennn in #17181 Add filename to GZIP stream error by @@marcoslot in #17166 Issue #17115: TimeTZ Approximate Quantile by @@hawkfish in #17162 Issue #17046: AsOf Left Predicates by @@hawkfish in #17159 [Fix] Pass delete indexes when committing updates by @@taniabogatsch in #17176 Python.yml: Add back logic to perform fast-fail on Python 3.10 by @@carlopi in #17107 Notify JDBC repo to run Vendor.yml workflow by @@staticlibs in #17099 Issue #17049: ICU Date Cast by @@hawkfish in #17067 Add bind_operator callback to TableFunction - allowing table functions to directly emit a LogicalOperator by @@Mytherin in #17196 [ENCRYPTION] Make block header size adaptive by @@ccfelius in #17118 Issue #16839: Disable TIMESTAMP Casts by @@hawkfish in #16899 Add support for an explicit PRESERVE_ORDER flag for copy to file by @@Mytherin in #17199 Add SYSTEM_PEAK_BUFFER_MANAGER_MEMORY and SYSTEM_PEAK_TEMP_DIRECTORY_SIZE to profiler by @@lnkuiper in #17164 Fix [InvokeCI / NotifyExternalRepository] Unexpected value 'true' by @@hmeriann in #17212 Add support for the cast_to_type function, that allows generating a cast from an expression to the type of another column by @@Mytherin in #17209 Better cardinality estimates for inequality joins/grouped aggregations by @@lnkuiper in #17139 Add ExternalFileCache validation as option for ExtendedOpenFileInfo by @@lnkuiper in #17205 Explicitly flush the thread-local optimistic writer in PhysicalBatchInsert when finalizing by @@Mytherin in #17214 Pushdown arbitrary expressions into scans by @@Mytherin in #17213 Fix #17170: sort selection result in OR expression by @@flashmouse in #17180 [Dev] Re-enable Iceberg, Bump Avro, fix generate_extension_functions.py for dependencies between extensions by @@Tishj in #17204 Change Invalid Unicode Error to Invalid Encoding by @@pdet in #17208 Direct IO for temp files by @@lnkuiper in #17219 Fix [InvokeCI / NotifyExternalRepository] GitHub Actions has encountered an internal error when running your job. by @@hmeriann in #17218 Add "thousands" option to CSV Reader by @@pdet in #17220 add capi functions to create map and union values by @@jraymakers in #17227 Only notify JDBC when all runs are successful by @@staticlibs in #17233 Update Friendlier SQL link.md by @@hfrifkin in #17248 Implement reading concatenated GZIP members by @@lnkuiper in #17255 Return invalid BufferHandle upon loading a destroyed BlockHandle by @@lnkuiper in #17249 Internal #4772: Timestamp Error Parameter by @@hawkfish in #17283 BUGFIX: do not perform unused columns optimization in presence of multiple grouping sets by @@Tmonster in #17259 Internal #4532: 13 Month Intervals by @@hawkfish in #17303 Dont try to load extension if storage type is already registered by @@Maxxen in #17241 Adapt size of hash table during aggregation using HyperLogLog by @@lnkuiper in #17236 Switch to always using list identifier instead of array by @@J-Meyers in #17242 Add root's query_location also to TransformInterval by @@carlopi in #17271 Histogram table function test by @@hmeriann in #17276 Guess Parquet footer size by @@lnkuiper in #17300 Issue #16563: FLOAT to DECIMAL by @@hawkfish in #17302 Feature #15873: Windowed ORDER BYs by @@hawkfish in #17304 Switch from Bottom-Up to Top-Down Decorrelation Strategy by @@kryonix in #17294 Generating random data for mbedtls without key by @@ccfelius in #17309 Fix CI by @@Mytherin in #17319 [Arrow] Implement support to consuming and producing Decimal 32 and 64. by @@pdet in #17314 take the column ids from the logical get, don't require a LogicalGet … by @@Tishj in #17315 Allow installing extensions with external access allowlist by @@samansmink in #17316 Implement ARTMerger replacing the recursive ART merge algorithm by @@taniabogatsch in #17243 Share null mask with constant null arg vector by @@iceTTTT in #17234 Fix #17311: correctly check for presence of recursive keys in transformer by @@Mytherin in #17320 [CSV Reader] Simplify Quote/Escape detection code, make it more robust and decouple comment and skip_rows option. by @@pdet in #17284 Fix try_cast from NaN double to decimal by @@lnkuiper in #17322 Add serialization for new TableColumn type by @@Mytherin in #17321 Extract expressions from nested conjunction AND for index scan by @@lnkuiper in #17297 Support late materialization in the Parquet reader, and handle COUNT(*) directly in the multi file reader by @@Mytherin in #17325 Implement ARTOperator replacing Lookup and the recursive Insert by @@taniabogatsch in #17327 Internal #4723: Inequality Condition Pushdown by @@hawkfish in #17317 Properly format strings when throw JSON errors by @@lnkuiper in #17331 Fix potential vulnerable cloned function by @@npt-1707 in #17340 Fix potential vulnerable cloned function by @@npt-1707 in #17339 Revert "Skip MinGW, currently failing on main" by @@carlopi in #17342 Unify Parquet Metadata cache invalidation logic with Cached File System cache invalidation by @@Mytherin in #17334 Fix issue with empty ranges by @@kryonix in #17332 Internal #4797: Timestamp Range Cardinality by @@hawkfish in #17330 Some nitpicking fixes by @@szarnyasg in #17337 Issue #17299: Integer Rounding by @@hawkfish in #17328 Parquet Reader: emit partition stats for any files that have cached metadata, and implement ListFilesExtended that adds extra info to files globbed by @@Mytherin in #17344 Add support for UUID v7 to Filename Pattern - and clean it up so that it correctly supports composite patterns by @@Mytherin in #17345 Add support for the HIVE_FILE_PATTERN option - that allows partitioned files to be written without writing them to a hive-style directory structure by @@Mytherin in #17346 Add an OnDetach callback to the catalog that is triggered when the user detaches a catalog by @@Mytherin in #17347 Pass commit ID to NotifyExternalRepositories.yml by @@staticlibs in #17333 Add support for BENCHMARK_ROOT_DIRECTORY cmake option to change benchmark runner root directory, and add support for cache_file and reload options to enable better caching for non-DuckDB databases by @@Mytherin in #17355 Support --directories option in format.py by @@Mytherin in #17354 Handle both ENCRYPTION_KEY and STORAGE_VERSION passed as options by @@carlopi in #17357 Fix internal exception from assigning invalid index to optional_idx query_id; by @@Tishj in #17359 Fixup amalgamation: reqlen is only used with assert enabled by @@carlopi in #17361 md5_number: return UHUGEINT by @@szarnyasg in #17336 Skip emitting partition stats if "has_deletes" is set in the file info by @@Mytherin in #17365 Benchmark runner: add argument, include and load_only options - and make ClickBench run the original benchmark instead of a subset by @@Mytherin in #17367 Fix two off-by-one errors in row estimate of range and generate_series by @@JelteF in #17373 [Nested] Fix: 16489 - Find NULLs in lists using list_position by @@maiadegraaf in #17080 fix #17258: Allow to open database in readonly mode within cli by @@jjballano in #17375 Join Hash Table Probing Optimization: Optional Probing Selection Vector by @@gropaul in #17062 Remove bundled TPCH & TPCDS in Python wheels by @@carlopi in #15923 [Compression] Introduce DICT_FSST compression method by @@Tishj in #15637 Deprecate lambda arrow (->) and replace it with LAMBDA x : x + 1 by @@taniabogatsch in #17235 fix not setting nested validity when map_extract returns null by @@Maxxen in #17379 Function chaining: report missing column instead of missing function if function exists by @@Mytherin in #17383 Improve error messages in UPDATE ... SET by @@Mytherin in #17384 Add candidates suggestion when COLUMNS regex does not match any columns by @@Mytherin in #17385 add step to clean up the disc space to fix No space left on device by @@hmeriann in #17390 Fix issue in string -> hugeint conversion with decimals and exponents by @@Mytherin in #17388 Improve error message reporting for cast failures by @@Mytherin in #17382 Fix Python CI: pin virtualenv to previous version by @@Mytherin in #17386 Improve error reporting for missing qualified columns by @@Mytherin in #17397 Issue #17266: Lead Lag Nulls by @@hawkfish in #17391 Fix #17266：the result of lad/lead when the offset is null by @@ditdb in #17268 VirtualFileSystem to take an input, allowing to customize behaviour by @@carlopi in #17393 [Dev] Add QualifiedName::ParseComponents, add input to the error messages by @@Tishj in #17403 Provide suggestions and a link to the documentation for OOM errors by @@Mytherin in #17402 [Dev] Flatten any deeper children vectors, when the top level is a FLAT vector by @@Tishj in #17387 Minor fixes for the CLI by @@Mytherin in #17405 Add support for CREATE OR REPLACE TYPE, CREATE TYPE IF NOT EXISTS and CREATE TEMPORARY TYPE by @@Mytherin in #17404 Use an insertion order preserving map in Value::MAP by @@taniabogatsch in #17389 Implement json_each/json_tree by @@lnkuiper in #17406 Fix #16552: adjust join condition sequence by @@flashmouse in #16943 WAL replay index fixes by @@taniabogatsch in #17409 ZSTD: use a high penalty when min size is exceeded instead of disabling compression to allow force compression to work by @@Mytherin in #17412 Internal #4723: PWMJ Inequality Pushdown by @@hawkfish in #17400 Move all httplib code to HTTPUtil class by @@Mytherin in #17420 Avoid generating default views and macros in the temporary catalog by @@Mytherin in #17408 unittest: improve detection of whether or not we can run --force-restart tests by @@Mytherin in #17419 Give tasks a TaskType with a name by @@Mytherin in #17421 Use argparse in scripts/format.py by @@adsharma in #17360 Add missing commas by @@szarnyasg in #17424 Internal #4830: IEJoin Inequality Pushdown by @@hawkfish in #17422 Add conn.query_progress() method by @@nickzoic in #16927 Fixes filter pruning use the statistics updated by the same filter by @@Damon07 in #17425 Fix JSON extension compilation on Ubuntu 22.04 by @@staticlibs in #17434 Use pytest in SQLLogic Python test runner by @@Flogex in #16685 On COPY TO/FROM check the format during binding. by @@pdet in #17381 BUGFIX: DELIM_JOINS should reflect functionality of NULL filtering conditions in joins with DELIM_GETS by @@Tmonster in #16910 Allow directly attaching of Parquet/CSV/JSON files by @@Mytherin in #17415 Force errors when trying lines as early as possible by @@pdet in #17427 Enable SYSTEM_PEAK_BUFFER_MEMORY and SYSTEM_PEAK_TEMP_DIR_SIZE profiling by default by @@lnkuiper in #17407 [C API] Expose the client context, connection id and scalar function bind data by @@taniabogatsch in #17449 [CSV Sniffer] Proper type replacement in header only files by @@pdet in #17447 Recurse into MAP and LIST with the remap_struct and the MFR ColumnMapper by @@Tishj in #17448 Fix: pyproject.toml does not contain a tool.setuptools_scm section by @@YUKI2eN3e in #17443 [Fix] Macro binding with unknown parameters in list_has_all and some other code tidying by @@taniabogatsch in #17450 Generalize HTTP interface and use the new HTTP interface in httpfs by @@Mytherin in #17464 [Fix] Switch between constant and flat vector in C API by @@taniabogatsch in #17465 Fix TIMETZ cast in example by @@szarnyasg in #17468 Remove duplicated arrow fetch test by @@emmanuel-ferdman in #17476 Multi File Reader Rework (Part 19): Make MultiFileReaderInterface virtual, and move reading methods to the BaseFileReader by @@Mytherin in #17475 [Serializer] Lambda Compatibilty Fix by @@maiadegraaf in #17428 fix parsing bool values in JSON by @@ccfelius in #17460 Emit dictionary vectors with unaligned start index by @@OmidAfroozeh in #17471 Add release version by @@hannes in #17479 Expose qualified table names in GetTableNames and add duckdb_get_table_names to C API by @@taniabogatsch in #17472 Bump avro, httpfs, mysql, postgres and sqlite by @@Mytherin in #17482 Fix GeoParquet ExpressionColumnReader schema by @@Maxxen in #17481 add regression_threshold_seconds argument to regression/test_runner.py by @@hmeriann in #17485 DROP of missing entry should fail in binding by @@jeewonhh in #17474 HTTPFS Parameters fix by @@Mytherin in #17486 HTTPUtil Fix: correctly pass in on_retry by @@Mytherin in #17494 Bump spatial & vss by @@Maxxen in #17492 Add support for altering structs (drop, add, rename field) inside LIST and MAP columns. by @@Tishj in #17462 [Python Dev] Guard against python exceptions when interacting with the currentframe object by @@Tishj in #17490 If distinct count from stats is 0, do not use it in Join Order Optimizer by @@Tmonster in #17466 Make the encodings extension a core extension, and make it auto-loadable. by @@pdet in #17206 Allow passing down rc-style version also via OVERRIDE_GIT_DESCRIBE by @@carlopi in #17501 Allow DUCKDB_EXPLICIT_VERSION to be propagated by @@carlopi in #17498 Minor nightly fixes by @@Mytherin in #17500 Add FileSystem::TryRemoveFile - that only removes a file if it exists by @@Mytherin in #17502 Add OperatorFinalize callback to operators - which is called after a pipeline is finished by @@Mytherin in #17503 Apply dynamic filter pushdown of TopN optimizer also to existing TopN nodes by @@Mytherin in #17504 Fix: Optional Probe Selection by @@gropaul in #17505 FileHandle Logging by @@samansmink in #16758 Fix typos by @@szarnyasg in #17478 Remove spatial from OSX Relassert by @@carlopi in #17509 Update more extensions by @@Maxxen in #17510 Bump HTTPFS again by @@Mytherin in #17511 feat: include catalog and schema names in function serialization by @@rustyconover in #17512 Fix encodings by @@carlopi in #17514 Fix python nightly build by @@Tishj in #17515 Use Catalog::TryAutoLoad for encodings extension by @@pdet in #17520 [Python Dev] Using reinterpret_steal breaks the refcount of the passed-in object by @@Tishj in #17525 Fix update extensions by @@carlopi in #17527 Minor fixes to exception error messages by @@carlopi in #17528 [Python Dev] Fix failing tests for the Python SQLLogicTester by @@Tishj in #17529 Resolve GitHub workflow set-output deprecation warnings by @@kurtmckee in #17516 [CSV Reader] Detect SQLNULL types for schema merging, use schema merging in csv relations, add files_to_sniff option. by @@pdet in #17467 Fix extension test by @@carlopi in #17536 [Dev] Fix crash when describing a table with a virtual column by @@Tishj in #17544 [HTTPUtil] Let requests made through the HTTPUtil interface accept URI's without a scheme. by @@Tishj in #17545 Attach after setting database type by @@Mytherin in #17546 Pass MultiFileGlobalState to InitializeReader, and pass file list to CreateMapping instead of eagerly getting the first file by @@Mytherin in #17553 [Dev] Fix allowed_directories crash by @@Tishj in #17548 [Fix] duplicate filters during index scans by @@taniabogatsch in #17547 Generate data for tpch sf100 in steps by @@Tmonster in #17539 Issue #17537: Fractional Second Padding by @@hawkfish in #17556 Make MultiFileList::Copy a virtual method by @@Mytherin in #17566 [Dev] Can't use USING COMPRESSION with a deprecated compression type by @@Tishj in #17542 Add (de)serialization for ExtraOperatorInfo by @@NiclasHaderer in #17563 Fix issue with ExternalFileCache when data is evicted by @@lnkuiper in #17567 Remote Reads: allocate correct buffer size for prefetch by @@Mytherin in #17557 Remove patch and bump httpfs by @@carlopi in #17558 [Dev] Fix Arrow fixed size binary reading by @@Tishj in #17573 Fix setup.py to correctly handle OVERRIDE_GIT_DESCRIBE by @@carlopi in #17580 @ text @d1 1 a1 1 # $NetBSD: Makefile,v 1.9 2025/05/20 16:03:33 ryoon Exp $ d3 1 @ 1.9 log @databases/duckdb: Update to 1.2.2 Changelog: 1.2.2: This is a bug fix release for various issues discovered after we released 1.2.1. There are no new major features, just bug fixes. Database files created by DuckDB versions all the way back to v0.9.* can be read by this version. What's Changed [Python] Fix deadlock in from_parquet with file_globs caused by not releasing the GIL by @@Tishj in #16522 [Python] Fix some stubs issues by @@Tishj in #16523 [Dev] Fix issues in the benchmark runner related to the serialized storage_version by @@Tishj in #16533 Set estimated cardinality to newly created logical operators by @@jeewonhh in #16528 custom_extension_repository to also be the default autoinstall_extension_repository by @@carlopi in #16459 [Python Dev] No longer trigger a DeprecationWarning when using a UDF by @@Tishj in #16433 Fixup problems connected to prep to 1.2.1 by @@carlopi in #16578 Give preference to quote=escape if we can't do better by @@pdet in #16584 Max ART key length by @@taniabogatsch in #16588 Issue #16617: Window Constant Finalize by @@hawkfish in #16628 [Fix] Index scan - Move the table scan state into the local state by @@taniabogatsch in #16650 [Fix] Perform eager constraint checking during physical batch insert by @@taniabogatsch in #16651 [Python] Don't push down OPTIONAL_FILTER into pyarrow for the arrow scan. by @@Tishj in #16657 move OrderPreservationRecursive to physical_plan_generator.hpp by @@jeewonhh in #16656 Add libs folder to bundled static libs by @@taniabogatsch in #16655 Avoid UMA by @@krlmlr in #16641 Avoid UMA by @@krlmlr in #16643 Avoid UMA by @@krlmlr in #16642 Several fixes for CSV fuzzer tests by @@pdet in #16636 FSST Fix: Correctly detect the situation where we have 3 bytes remaining by @@Mytherin in #16688 Fix #16627: handle DISTINCT FROM and NOT DISTINCT FROM in zone-map pushdown by @@Mytherin in #16691 Fix #16554: emit blobs as part of .dump by @@Mytherin in #16693 add avro by @@samansmink in #16708 Issue #16652: Implicit Ordered Aggregation by @@hawkfish in #16718 Issue #16649: SelectNth Remainders by @@hawkfish in #16705 [Dev] Fix EXPORT DATABASE missing semicolons in produced schema.sql by @@Tishj in #16723 Always throw the error that happens the earliest in the CSV Reader by @@pdet in #16728 Fix #16662 by @@lnkuiper in #16732 [Test] Add missing test for eager-constraint checking fix by @@taniabogatsch in #16745 Clamp reported memory in duckdb_memory by @@Mytherin in #16753 CLI -help: Fix default value for -nullvalue by @@carlopi in #16755 Apply RLE fix to v1.2 by @@Mytherin in #16756 [Arrow] Setting schema of the keys to not null for maps and properly write Null columns to arrow by @@pdet in #16711 Fix min/max values in numeric cast exception message by @@abramk in #16777 [ADBC] Add wrapper to connection new, set options at connection init, if any. by @@pdet in #16748 Remove delta from extensions built on a nightly basis (vs 1.2-histrionicus) by @@carlopi in #16794 PhysicalTopN: Buffer-allocated StringHeap by @@lnkuiper in #16770 [chore] Add v1.2.2 to storage versions, preparation for upcoming patch release by @@carlopi in #16799 Fix issue when line is empty by @@pdet in #16823 Add extra check in normalize for commit, rollback and abort by @@pdet in #16802 Fix #16783: Fix DistributivityRule by @@flashmouse in #16804 Internal #4492: Ignore Nulls Orrification by @@hawkfish in #16837 Fix #16836: rewrite main column data in case of an update that only modifies the validity by @@Mytherin in #16851 Initialize type by @@pdet in #16848 [CSV Reader] Add check on ever quoted for candidate selection by @@pdet in #16868 Clean up partial deletes when encountering a transaction conflict in a vector by @@Mytherin in #16905 Fix clang-tidy: add missing include by @@Mytherin in #16920 Backport "Adding gzip version of shell for linux/osx install script" by @@carlopi in #16917 Backport "Adding gzip version of shell for linux/osx install script"/2 by @@carlopi in #16924 Skip tests for Python 3.8, already end-of-life by @@carlopi in #16923 Bump to avro version that now support wasm by @@carlopi in #16916 Unwrap DUCKDB_EXTENSION__LINKED_LIBS via CMake by @@carlopi in #16914 Skip manylinux-extensions-x64 job by @@carlopi in #16932 [Fix] Disable recursion for invalidated database error by @@taniabogatsch in #16907 bump excel (on v1.2) by @@Maxxen in #16915 [Fix] Flush local storage before any deletes by @@taniabogatsch in #16951 [Python] Fix PyArrow filter pushdown for NaN by @@Tishj in #16952 Allow overriding the Printer::Print output destination by @@bleskes in #16909 bump out-of-tree extensions to v1.2.2 by @@Tmonster in #16913 Allow SQL prepared statements to be rebound by @@NiclasHaderer in #16820 Fix #16959: binding constraints should not modify the original constraints by @@Mytherin in #16971 Add back manylinux extensions [on v1.2-histrionicus] AND bump extensions by @@carlopi in #16976 Skip TPC-DS Q67 on DOUBLE by @@Mytherin in #16987 Re-enable iceberg by @@carlopi in #16992 Try enabling and bump delta by @@Maxxen in #16990 Backport @@szarnyasg's 16999 by @@carlopi in #17013 1.2.1: This is a bug fix release for various issues discovered after we released 1.2.0. There are no new major features, just bug fixes. Database files created by DuckDB versions all the way back to v0.9.* can be read by this version. What's Changed [Dev] MultiFileReader fix InternalError in CreateFilterMap by @@Tishj in #16114 [bug-fix] Avoid throwing in catch block for failed commits by @@Vegetable26 in #15903 Issue #16098: ValidEnd Parallel Vectorisation by @@hawkfish in #16140 Adding an extension option shouldn't delete a set value and promote unrecognized options by @@bleskes in #15919 Parquet writer: Re-implement GetRowSize for Strings by @@Mytherin in #16178 Fix #16157: correctly get the first row when reading hive partitions from a dictionary vector by @@Mytherin in #16180 Fix #16122: bind default values in a sub-binder to avoid modifying the catalog search path of the current binder by @@Mytherin in #16181 Fix #16134: when a catalog/schema/table has the same name, we prefer to suggest the table name by @@Mytherin in #16189 Ensure MergeCollectionTask has a writer by @@ywelsch in #16207 Backport #16115 by @@NiclasHaderer in #16227 Deleted copy constructor of pending query by @@NiclasHaderer in #16242 Correctly report errors caused by get_database in C extensions by @@mlafeldt in #16253 use random seeds for bernoulli sample when parallel is enabled by @@Tmonster in #16223 Parquet Reader: avoid caching the compressed buffer in the ColumnReader by @@Mytherin in #16263 Fix #16260: correctly handle parameters in getvariable by @@Mytherin in #16264 Avoid calling SetFilterAlwaysTrue multiple times in RowGroup::CheckZonemap by @@Mytherin in #16269 [Fix] Scanning from normal leaf to nested leaf by @@taniabogatsch in #16270 Fix #16231: refer to order by condition in ARRAY(SUBQUERY) by alias instead of by index by @@Mytherin in #16272 Fix #16257 by @@lnkuiper in #16275 AFL Tests for the CSV reader by @@pdet in #16280 Issue #16250: Window Range Performance by @@hawkfish in #16276 Fix #16278: late materialization should not trigger on very large limits, and it should never trigger on limits without offsets when preserve_insertion_order = false by @@Mytherin in #16282 Overflow strings allocations: avoid rounding up memory allocated per overflow string - when reading "small" overflow strings place them directly into the vector instead by @@Mytherin in #16283 Use ordered map to preserve expressions order by @@Damon07 in #16111 [Dev] register_filesystem stubs, use fsspec.AbstractFileSystem, not str by @@Tishj in #16266 [Python Dev] Fix crash with empty args for isin | Fix transformation for isnotin by @@Tishj in #16271 Fix issue related to hang when all candidates are eliminated in refinement by @@pdet in #16288 [Fix] Early-out on CREATE INDEX (IF NOT EXISTS) by @@taniabogatsch in #16093 [Python] Fix the reverse binary expressions in the Expression API by @@Tishj in #16300 Bugfixes by @@lnkuiper in #16244 [Fix] MinGW bundle static libs by @@taniabogatsch in #16292 Fix heap buffer oveflow sampling by @@Tmonster in #16279 Expose STRING_LITERAL in C API by @@taniabogatsch in #16293 Allow accessing profiler query tree under lock by @@ywelsch in #16314 bump extensions by @@Maxxen in #16313 Inline virtual list lambda bind functions by @@Maxxen in #16327 Update shell.cpp to fix #16333 by @@teaguesterling in #16335 Add the suggestion to verify the nullstring as part of the cast error message by @@pdet in #16336 Take NULLs into account for DELTA_BINARY_PACKED by @@lnkuiper in #16317 PhysicalTableScan: Adapt to allow async behaviour by @@carlopi in #16310 Allow querying attached catalog from detached catalog by @@jeewonhh in #16289 Reduce minimum expected memory usage in RadixPartitionedHashTable by @@lnkuiper in #16332 Backport ebb4dcc to v1.2-, adding missing include by @@carlopi in #16369 Excecption load on mismatched ABI: Use '%d' to print ABI type by @@carlopi in #16367 Aggregation: For dictionaries without an id - use the correct threshold to bail-out on using the dictionary by @@Mytherin in #16364 Bump excel to the same version distributed on core repository by @@carlopi in #16375 Accept valid dialects with escape set into the refinement phase by @@pdet in #16387 Push the correct casts for values of different types in (X, Y) IN (SELECT X, Y) by @@Mytherin in #16392 Add support for autoload and autoinstall for ui extension by @@carlopi in #16393 Add twine_upload option to Python.yml to trigger upload by @@carlopi in #16410 Only select options that generate more columns with null_padding, if they at least hold 50% of consistency by @@pdet in #16404 Use checkpoint bind in DuckTableEntry::Copy to avoid re-validating default values (and potentially causing issues during WAL replay) by @@Mytherin in #16398 Rename DUCKDB_API to DUCKDB_C_API for duckdb.h by @@Mytherin in #16397 Issue #16407: Try_Strptime Invalid TimeZone by @@hawkfish in #16416 Internal #4303: Windowed DISTINCT Leaks by @@hawkfish in #16417 Internal #4258: MODE Spooling Stability by @@hawkfish in #16415 Fix PyPi upload also for branches, when twine_upload is provided by @@carlopi in #16421 [Fix] Throw constraint violation for FK constraint checking by @@taniabogat Add storage and serialization version for v1.2.1 by @@Flogex in #16403 Update flaky return_files.test by @@jeewonhh in #16432 Add python version to duckdb_api by @@guenp in #15599 Do not accept null values in lists for column parameters by @@pdetigning using azure by @@hannes in #16444 CSV small code Improvements + initialising boolean variable. by @@pdet in #16454 [Python Dev] Make pandas not required in a couple places, check if it's installed in others by @@Tishj in #16414 fix passinger in #16480 [Python Dev] pyproject.toml should not use oldest-supported-numpy anymore by @@Tishj in #16486 [tests] Multiple FORMAT in copy, only last one matters by @@carlopi in #16493 Bump postgres_scanner and fts extensions by @@lnkuiper in #16492 bump sqlsmith extension tag by @@Tmonster in #16488 [BugFix]: Swap join children, not left and right set by @@Tmonster in #16487 [tests] Add allow_unsigned_extensions require by @@carlopi in #16499 Provide callback when tasks are starting / stopping by @@ywelsch in #16451 CodeQuality: ubuntu-20 to ubuntu-22, lock black to version 24 and trick clang_format detection by @@carlopi in #16513 Move from ubuntu-20:04 to ubuntu-22:04, part I by @@carlopi in #16510 [chore] Build Linux releases also on PRs AND ubuntu-20 to 22 by @@carlopi in #16506 Move from ubuntu-20:04 to ubuntu-22:04, part II by @@carlopi in #16514 bump spatial and excel for v1.2 by @@Maxxen in #16504 CI Fixes after upgrade to ubuntu 22 by @@Mytherin in #16516 LinuxRelease.yml: Pass down override git describe by @@carlopi in #16521 bump spatial again by @@Maxxen in #16518 @ text @d1 1 a1 1 # $NetBSD: Makefile,v 1.8 2025/02/26 15:42:51 ryoon Exp $ d3 1 a3 5 DISTNAME= duckdb-1.2.2 CATEGORIES= databases MASTER_SITES= ${MASTER_SITE_GITHUB:=duckdb/} GITHUB_PROJECT= duckdb GITHUB_TAG= v${PKGVERSION_NOREV} d19 2 d24 1 @ 1.8 log @databases/duckdb: Update to 1.2.0 Changelog: 1.2.0: What's Changed Optimise division by a constant at runtime for integer division by JAicewizard in 10348 Add cross join to Python Relational and PySpark API by khalidmammadov in 13519 Fix 13805: throw a more descriptive error message when an on-disk file is referenced using a replacement scan for an unsupported file format by Mytherin in 13871 Make sampling accept parameters at the parser/transformer layer by Mytherin in 13903 Fix 13867: use 64-bit random numbers to generate random numbers for random() by Mytherin in 13920 Fix 13769: when binding views, always first search in the schema that the view is defined in by Mytherin in 13921 Rework table bindings to be components (catalog, schema, table) instead of flat strings by Mytherin in 14017 Add auto-loadable extension settings to duckdb_config_count and duckdb_get_config_flag by Mytherin in 14021 Fix 10961 - in the HAVING clause - in case of column name conflicts, bind to aliases instead of to ungrouped columns by Mytherin in 14023 Enable filter pushdown through Logical Unnest by Tmonster in 14008 Allow duplicate table aliases in the table binder by Mytherin in 14035 Unify DESCRIBE [query] and DESCRIBE [table] by Mytherin in 14039 Support qualified identifiers in the EXCLUDE clause by Mytherin in 14043 Add SMALLER_BINARY flag to reduce binary size by Mytherin in 14057 Smaller Binary: remove more templates from arg_min_max by Mytherin in 14071 Unify entropy and mode aggregates - and skip specialized implementations for entropy with smaller binary by Mytherin in 14080 [Python] Add set_default_connection to the duckdb module by Tishj in 13442 Provide workaround for prefetching parquet files with incorrect page offsets by samansmink in 13697 Move core_functions to a separate extension by Mytherin in 14149 PySpark df.drop() to support expressions by khalidmammadov in 14059 add some RealNest benchmarks by hmeriann in 13345 feed table function into multifilereader initialization by samansmink in 14112 [Dev] Fix an issue causing ExecuteTask to do much more work than intended by Tishj in 14034 Overhaul Parquet dictionary handling by hannes in 14194 [Feature] Allow passing the catalog (database name) to appender by taniabogatsch in 13692 Add Taxi Dataset Benchmark by pdet in 14197 Feature 3036: Window Spooling by hawkfish in 14181 Small C Extension API changes by samansmink in 13987 Add HTML and Graphviz support for explain analyze by abramk in 13942 Fix 13064: offer more suggestions with same score by Damon07 in 14048 New Algorithm to find a new line on parallel execution by pdet in 14260 Making client context lock optional for relation binding by pdet in 14093 [Feature] Allow passing the catalog during C API appender creation by taniabogatsch in 14256 Make test random output ordered by Damon07 in 14267 Skip test_window_distinct by Mytherin in 14309 Taxi Benchmark by pdet in 14301 Switch to shared pointer for multfilelists by samansmink in 14291 Push 14298 to feature branch by flashmouse in 14311 Implement PullUp Empty Results optimizer by Tmonster in 13524 [Export/Import] Use the DependencyManager to (stable) sort the entries before export by Tishj in 14196 Partitioning-Aware Aggregation and Partitioning-Aware Infrastructure by Mytherin in 14329 Add df.unionByName to PySpark API by khalidmammadov in 14063 Or filter pushdown into zone maps by Tmonster in 14313 Get the current setting in the database file opener by Mytherin in 14361 [Feature + Fix] Support ALTER TABLE tbl ALTER col TYPE USING and fix null handling in struct_insert by taniabogatsch in 14359 [C API] Add table_description_create_ext and table_description_get_column_name by taniabogatsch in 14285 Move _rtools platform to be equivalent to _mingw by carlopi in 14368 Fix for accidental like skip in the CSV Buffer by pdet in 14380 Table locks - always grab table locks through the transaction interface by Mytherin in 14379 Implementing array_slice and [] for BLOB by hannes in 14358 Rework settings handling and implement auto-generation for new ones by Mytherin in 14383 Rework settings handling and implement auto-generation for new ones by chrisiou in 14018 Arrow list buffer - suggest setting arrow_large_buffer_size to true when regular list buffer size is exceeded by Mytherin in 14384 Fix incorrect merge conflict resolution in workflow file by Mytherin in 14390 Update Parquet Thrift to latest version by hannes in 14258 Reformat list functions by c-herrewijn in 14372 Tidy Check to do complete run also on feature by carlopi in 14394 [Python] Use an ArrowQueryResult in FetchArrowTable when possible. by Tishj in 14319 Make mysql_scanner auto-loadable, and add mysql/postgres secrets by Mytherin in 14392 Improvement the speed of table sample systems by continue-revolution in 12631 Support defining column names in CTAS by douenergy in 14327 Fix pointer indirection in pyrelation.cpp by carlopi in 14403 Fix idx_t to int64_t implicit conversion flagged by clang-tidy by carlopi in 14402 Storage: make ROW_Gp-N: Rework to use heap of sort keys by Mytherin in 14424 reformat string functions by c-herrewijn in 14400 Prefix Aliases in SQL by hannes in 14436 [Dev] Optimize ValidityMask when reading from a ColumnDataCollection by Tishj in 14416amically generated join filters through UNION, UNNEST and AGGREGATE by Mytherin in 14453 Try auto-casting for mismatching data chunks in the Appender API by taniabogatsch in 14433 Implement DELTA_BINARY_PACKED compression in Parquet writer by om C to C++ by Mytherin in 14473 Fixing an issue with parquet dictionary reading by hannes in 14438 Strip down unused/unsupported options from the CLI by Mytherin in 14478 [PySpark] Add withColumns, withColumnsRenamed, cos, acos, any_valState by Mytherin in 14483 CLI Code Cleanup: Move rendering logic into separate Renderer classes by Mytherin in 14485 Reformat compressed materialization functions by c-herrewijn in 14470 Internal 3273: Shared Window Expressions by hawel Reading Validation by pdet in 14439 Avoid recompilations of duckdb when there are no actual changes by carlopi in 14176 Add -safe mode to shell which disables external access, and remove SQLite UDFs from the shell by Mytherin in 14509 [PySpark] Add functions covar_pop, covar_samp, call_functions, endswith, startswith, exp, factorial, log2, ln, degrees, radians, atan, atan2, tan, round, bround by binste in 14454 Reformat arithmetic operators by c-herrewijn in 14489 add attach with default tables by samansmink in 14118 Add duckdb_param_logical_type by Giorgi in 14515 Remove most BUILD_ options for extensions, using CORE_EXTENSIONS by carlopi in 14531 CLI: more code clean-up by Mytherin in 14551 Reformat nested and sequence functions by c-herrewijn in 14495 Parquet: Fixing selection vector calculation by hannes in 14558 CLI: Fix for .mode markdown rendering after refactor by Mytherin in 14569 Out-Of-Core Updates & Deletes by Mytherin in 14559 Manage enable_external_access at the FileSystem level, and add allowed_paths and allowed_directories option by Mytherin in 14568 feat(iejoin): use sort to replace binary search in iejoin by my-vegetable-has-exploded in 14507 Clean-up distinct statistics - add hashes cache add the Append and Vacuum layers, and remove unnecessary lock by Mytherin in 14578 [PySpark] Test Spark API with actual PySpark as backend by binste in 14526 Internal 3273: Shared Window Frames by hawkfish in 14544 Reformat aggregate functions by c-herrewijn in 14530 Expose threshold argument of Jaro-Winkler similarity by zmbc in 12079 No pushing filters below projections that cast to a lower logical type id by Tmonster in 13617 Implement left_projection_map for joins by lnkuiper in 13729 remove superfluous comment by c-herrewijn in 14586 [Dev] Make the regression_test_runner easier to replicate by Tishj in 14557 [PySpark] Add dataframe methods drop_duplicates, intersectAll, exceptAll, toArrow by binste in 14458 Internal 3381: Window Race Condition by hawkfish in 14599 Rework generated EnumUtil code by Mytherin in 14391 Force aggregate state to be trivially_destructible, unless AggregateDestructorType::LEGACY is used by Mytherin in 14615 AWS - remove expected error message by Mytherin in 14633 Temp directory compression by lnkuiper in 14465 Add support for SELECT * RENAME by Mytherin in 14650 [PySpark] Add autocompletion for column names to dataframes by binste in 14577 Force aggregate state to be is_trivially_move_constructible by lnkuiper in 14640 Correctly render EXPLAIN EXECUTE - use op.GetChildren() instead of hard-coding special cases by Mytherin in 14651 Buffer Manager - Make DestroyBufferUpon atomic by Mytherin in 14656 proposed enhancements to the query graphs by peterboncz in 14637 Sampling respects seed from random number generator if no seed is given. by Tmonster in 14374 Blockwise NL Join: Return control on every iteration in ExecuteInternal by Mytherin in 14658 feature(spark): add hex and unhex functions by spenrose in 14573 Support SELECT * LIKE '%col%' syntax by Mytherin in 14662 feature(spark): add base64 and unbase64 function by spenrose in 14561 Fix 14663: correctly propagate null values in list concat operator by Mytherin in 14675 ALTER TABLE ADD PRIMARY KEY by taniabogatsch in 14419 Merge feature into main by Mytherin in 14690 Support for CSV Encoding (UTF-16 and Latin-1) by pdet in 14560 Fix 14699 - Correctly handle SHOW TABLES in views by Mytherin in 14705 Fix 14701 - avoid flattening in-place in ColumnData Append method by Mytherin in 14708 Use TryCastAs instead of DefaultTryCastAs in comparison_simplification by Mytherin in 14711 Value interface & serialization clean-up by Mytherin in 14710 Fix various nightly CI issues by Mytherin in 14720 CLI: Add support for .thousand_sep and .decimal_sep by Mytherin in 14721 Propagate collations through functions in a generic manner by Mytherin in 14717 Add functions for handling null duckdb_values by Giorgi in 14687 adaptive filters should not reorder filters that can throw by Tmonster in 14672 [Python] Add LambdaExpression to the Python Expression API by Tishj in 14713 Add fallback for thread count if jemalloc cannot identify by lnkuiper in 14688 csv: parse escape character in unquoted fields by fanyang01 in 14464 [Python][Expression API] Add the between method on the Expression class by Tishj in 14726 [Attach][Macro] Fix issues identified with an attached macro by Tishj in 14715 Dont quote strings on csv files if quote='' by pdet in 14731 sqlite3_api_wrapper: avoid nullptr dereference by ProjectMutilation in 14748 Rework BlockHandle to no longer have friend classes, and rework ConvertToPersistent so it fails if there are active outstanding pins by Mytherin in 14746 Revert "CMake: Avoid dependency-inducing codegeneration of extension headers" by carlopi in 14723 [PySpark] Add more functions such as ascii, asin, btrim, char, corr, ... and fix differences in ordering of null values between PySpark and DuckDB by binste in 14738 Added list value getters duckdb_get_list_child and duckdb_get_list_size by prashanthellina in 14714 [Python][Expression API] Add collate to create a CollateExpression by Tishj in 14749 copy to operator still write schema for empty rows by wenjun93 in 14524 [Python] Use nullable dtypes in Pandas DataFrame creation when possible by Tishj in 14377 Update metrics generation script and include it in CI run by taniabogatsch in 14756 Add support for projection pushdown into struct fields by Mytherin in 14750 Optimistic writes: flush the last row group in all scenarios by Mytherin in 14759 Improve SqlStatement::ToString for UPDATE and DELETE statement to include alias of RETURNING clause by HarshLunagariya in 14765 Add JSON Logical Type metadata to parquet writer by niger-prequel in 14747 [Python] Add support for Expression to values to create a ValueRelation by Tishj in 14757 Add missing global options to Python's write_parquet by fr3fou in 14766 Add operator name to profiling output by ywelsch in 14744 Detect catalog changes on DROP IF EXISTS by ywelsch in 14742 Correctly deal with continued operation after reading a truncated WAL, and clean up WAL handling logic in storage manager by Mytherin in 14785 [Fix] Error message in transaction manager by taniabogatsch in 14788 Initialize the grouping sets when there is a group by all to enable filter pushdown by Tmonster in 14660 Merge feature into main again by Mytherin in 14793 [Python][Expression API] Add update to DuckDBPyRelation, accepting Expression objects | Add DefaultExpression by Tishj in 14780 Fix 14540: fix unnest rewriter by flashmouse in 14784 [PySpark] Add approxCountDistinct, add_months, and various array functions by binste in 14620 Add syntax highlighting support for errors in the CLI by Mytherin in 14799 Implement 14787: allow expressions in the aggregate clause of a PIVOT statement, as long as the aggregate clause only modifies the aggregate result and does not contain other columns by Mytherin in 14800 When repeatable is set, set ParallelSink to false by Tmonster in 14797 [Catalog] Fix issue related to uncaught problems during a COMMIT by Tishj in 14150 [Upsert] Support non-distinct values in the inserted data by Tishj in 14293 Fix issue copying a TABLE that references a SEQUENCE by Tishj in 14693 fix duckdb_extension.h macros for C by samansmink in 14808 LTO CMake setting was not working anymore on MacOS, fixing that by carlopi in 14811 Add syntax highlighting support to the DuckBox query result by Mytherin in 14820 Avoiding unnecessary rebinding by samansmink in 14616 Support struct projection pushdown in Parquet files by Mytherin in 14839 Internal 3263: Window Distinct Deadlock by hawkfish in 14775 Issue 14737: DISTINCT ORDER Dependency by hawkfish in 14840 [Python][Dev] Skip test_pandas_selection on Python3.8 by Tishj in 14851 [Python][Dev] Fix issues with new/updated tests in the python sqllogictest implementation by Tishj in 14850 add function ends_with back by Damon07 in 14859 Require capacity in ValidityMask by Mytherin in 14846 Issue 11557: DECIMAL Downcast Rounding by hawkfish in 14860 Increase map inference threshold by lnkuiper in 14848 Output exception message on parse exception by ackxolotl in 14852 Use LogicalTypeId::Unknown instead of LogicalTypeId::SQLNULL for macro binding by lnkuiper in 14809 return InsertionOrderPreservingMap from TableFunction to_string by samansmink in 14835 Support default values when appending data chunks by taniabogatsch in 14733 [PySpark] Add a lot more functions incl. some regexp ones by binste in 14761 Added getters for enum and struct type values by prashanthellina in 14831 Fix write partition columns false by ykskb in 14871 Generate In-Clause filters from hash joins by Mytherin in 14864 Move FTS extension out-of-tree by lnkuiper in 14872 [C API] More tests and nits by taniabogatsch in 14758 Issue 14885: DATEPART Cache Bounds by hawkfish in 14891 Fix arrow table filters by Tmonster in 14893 [Python] Fix various issues uncovered by 12959 by Tishj in 13149 Remove some Snappy definitions by lnkuiper in 14897 [Fix] Binder exception when creating a foreign key on a view by taniabogatsch in 14882 [C API] Implement AddColumn and ClearColumns for the Appender by taniabogatsch in 14880 python: use PyUnicode_FromStringAndSize() by methane in 14895 Top-N: Improve performance with large heaps, and correctly call Reduce by Mytherin in 14900 Append to child column first in list column append by Mytherin in 14902 Update cardinality during limit pushdown by jeewonhh in 14901 Add struct_concat by Maxxen in 14853 [Compression] Add ZSTD compression by Tishj in 14514 Improve timestamp functionality by taniabogatsch in 14818 Fix 14833: split_part follow pg by flashmouse in 14875 C API: Add Value Relation constructor with RelationContextWrapper and ParsedExpression as argument by anshuldata in 14892 Issue 14734: Wrap Parquet TIMETZ by hawkfish in 14908 [Fix] release shared connection pointer before it goes out of scope by roj516 in 14926 [Fix] Nightly async build by taniabogatsch in 14913 [Tests] Re-enable test for vector verification run by taniabogatsch in 14911 Return timestamp with timezone in read_text/read_blob by Maxxen in 14925 Fix several CLI issues by Mytherin in 14929 improve ReadAheadBuffer::AddReadHead error message by stephaniewang526 in 14940 Skip Dynamic Join Ordering Algorithm if there are many relations by Tmonster in 14943 remove failing benchmark by hmeriann in 14945 Typo in csv UnterminatedQuotesError how_to_fix_it by bradleybuda in 14951 Pullup empty results through delim joins as well by Tmonster in 14920 Fix getting named parameter type information. by Giorgi in 14952 Fix casting long to int via explicit cast in parquet by carlopi in 14959 Fix script/regression/benchmark.py rework by carlopi in 14958 Explicit install of pkg-config broke, removing it by carlopi in 14965 Improve code generation of storage and serialization version infos by carlopi in 14947 C API support for non-standard timestamp values by jraymakers in 14954 Implement Logical Compaction in Hash Join Operator by YimingQiao in 14956 Disable row group size bytes default initialization by lnkuiper in 14974 [Swift.yml] Bump to macos-14, and switch simulation targets by carlopi in 14984 Use IOException for failed fstat calls by ywelsch in 14975 Logical Sample requires child to have separate join order optimization by Tmonster in 14969 Properly register successful dialect runs by pdet in 14977 Run containerized builds requiring deprecatd ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION only on main/feature by carlopi in 14998 Fuzzer 3297: Nth Value Indexing by hawkfish in 14997 [Arrow] Filter pushdown decimal fix by Tishj in 14995 Support multiple function descriptions by c-herrewijn in 14838 Join Filter Pushdown does not push down in filters when nulls are present by Tmonster in 14970 [Fix] Throw on invalid MAP input in Value::MAP by taniabogatsch in 14916 Rely on extension-ci-tools workflow to build linux_amd64_gcc4 extensions by carlopi in 14987 Rework Auto-Complete To Work Based On PEG grammar by Mytherin in 15003 for-loop-erase bugfix in filter pushdown by peterboncz in 15008 Internal 861: Window Code Refactoring by hawkfish in 15007 Internal 3574: INTERVAL Normlisation Carries by hawkfish in 15009 [Arrow] Fix scan of an object providing the PyCapsuleInterface when projection pushdown is possible. by Tishj in 14993 [PySpark] - Add extra str functions to pyspark api by mariotaddeucci in 14921 [PySpark] - Add .isNull and .isNotNull methods to Column class by mariotaddeucci in 14960 DuckDB Arrow Non Canonical Extensions to use arrow.opaque by pdet in 15002 Autocomplete test fix by Mytherin in 15019 Add check_peg_parser to extension_entries by carlopi in 15021 Re-enable jemalloc on ARM by lnkuiper in 14810 Dynamically decide whether to do a Perfect Hash Join by lnkuiper in 14971 No salt for Android by lnkuiper in 14923 Fixup linux_arm64 extension builds by carlopi in 15016 Issue 14834: INTERVAL Collations by hawkfish in 15022 SUM(x + C) rewrite by Mytherin in 15017 Spell NULL with uppercase in configuration description and comments by szarnyasg in 15006 Force download doesn't require to do a head request by pdet in 14979 CSV Reader - 4 byte delimiters by pdet in 14670 More regression tests by lnkuiper in 14973 [PySpark] Add more functions such as slice, split, standard deviations, etc. by binste in 14863 Fix extension entries by Mytherin in 15027 Speed up scans of RLE compressed data by Mytherin in 15023 Speed up scans of Uncompressed strings by Mytherin in 15024 Internal 3583: INGNORE NULLS Race by hawkfish in 15032 [Regression.yml] Add icu, needed for external regression tests by carlopi in 15044 Fix internal error of list_zip with only truncate argument provided by Damon07 in 15039 Avoid sum rewrite for hugeint/uhugeint since it could introduce overflow errors by Mytherin in 15040 BarScalarFunction needs to keep track of width != string.size() by carlopi in 15041 Add SUM(BOOL) overload by Mytherin in 15042 Add virtual callback to get dependency manager to the catalog by Mytherin in 15043 Flip OR filter comparison if constant is on the other side by Mytherin in 15045 Fix 15010: in map cast only access validity when child elements were not fully converted by Mytherin in 15046 Various fixes for vector size = 2 CI by Mytherin in 15047 Add require ram to test runner, and use to limit distinct_grouping_tpch.test by Mytherin in 15048 [pystubs] Fix type of proto parameter in from_substrait methods. by ingomueller-net in 15004 CLI: Add -f [FILE] argument that allows execution of a file by Mytherin in 15050 max_temp_directory_size - print "90% of available disk space" as value if temp directory is not initialized by Mytherin in 15057 Interrupt query on error in ClientContext::Query by Mytherin in 15058 Turn count_if into an actual aggregate function by Mytherin in 15061 CLI: Add .safe_mode as a dot command as well by Mytherin in 15064 Pushdown inequality filtersy Mytherin in 15059 Restore support for DEBUG_STACKTRACE by carlopi in 15053 Shell: Provide a summary of numbers if we are rendering only a single row by Mytherin in 15031 Issue 15067: Postgres Age Compatibility by hawkfish in 15070 in quoted values. by pdet in 15018 IEJoin GetProgress: Normalize to 0-100 by carlopi in 15081 Avoid building for Python 3.7 on Windows by carlopi in 15085 Allow inputting a base hash in Regression workflow by lnkuiper in 15082 Topctly match only Sum aggregations in sum rewriter by Mytherin in 15088 new answers for some JOIN benchmarks by hmeriann in 15090 Ensure checkpoint tasks complete on IO exceptions by ywelsch in 15089 Internal 3615: Quantile Cursor Allocatin in 15303 remove conditional around fsync in single_file_block_manager by jkub in 15306 Move away from upload-artifactsv3 / download-artifactsv3 by carlopi in 15309 Fix update_extensions_ci test by carlopi in 15310 Addressing over-eager constraint checking with delete indexes by taniabogatsch in 15092 Fix internal issue 3740 by hannes in 15320 EXPLAIN/EXPLAIN ANALYZE - limit max lines of each extra info element, instead of truncating the entire node by Mytherin in 15317 Minor nightly test fixes by Mytherin in 15318 Bump Extension C API to stable by samansmink in 14992 Pass down DUCKDB_EXTENSION_SIGNING_PK as env by carlopi in 15324 Bump to latest sqlsmith and re-enable wasm by carlopi in 15323 Skipping lookups in GroupedAggregateHashTable if (almost) everything is unique by lnkuiper in 15321 Add automatic sampling regression fix 2 by Tmonster in 14914 [Dev] Fix Roaring compression bug on appending small vectors by Tishj in 15326 Fix JSON reader hang by lnkuiper in 15328 [Dev] Clean up Dictionary compression code by Tishj in 15300 Adjustments on test to bypass sniffing limitation on vector_size by pdet in 15330 Enable stack traces by default, split into getting the frame pointers and resolve symbols only when the error is finalized, and add support for demangling by Mytherin in 15337 Use correct element rename_list_el in grammar by Mytherin in 15339 Unified use of constant MainHeader: FLAG_COUNT by guoxiangCN in 15338 Append default to appender by Giorgi in 15121 add core functions make_timestamp_ns(nanos) and epoch_ns(timestamp_ts) by andreimatei in 14930 feat: support create_on_conflict in create_table_relation by scgkiran in 15245 Fix error message checking in test concurrent index by Mytherin in 15340 CI: Use mirror for Spark binaries by szarnyasg in 15372 Fix skip CSV Rejects test by pdet in 15359 Vectorize lookups in GroupedAggregateHashTable by lnkuiper in 15368 Bump azure and remove patches by carlopi in 15382 Fix conditional jump or move depends on uninitialised value(s) by pdet in 15367 Start encapsulating BaseExpression by Maxxen in 15360 [Python] Allow use of DuckDBPyType as child objects in implicit conversions by Tishj in 15346 [Dev] Made reference function private in ColumnSegment by Tishj in 15347 [Dev] Fix erroneous assert in ZSTD scan for LogicalTypeId::VARCHAR by Tishj in 15357 [Dev] Reset to the vector cache so the vectors are clean for the scan by Tishj in 15383 Fix tests not to use compatibility version latest by carlopi in 15361 Fix Test introduced by new sampling by Tmonster in 15378 Feature 12699: RANK Secondary Sorts by hawkfish in 15331 [Fix] Uninitialised values in list_reverse by taniabogatsch in 15387 [Dev] Check in insert if the InsertionOrderPreservingMap contains the key, do nothing in that case by Tishj in 15385 AFL++ Fuzzer Tests and Fixes by pdet in 15329 Fix RelationStatisticsHelper to estimate table filters correctly by Tmonster in 15308 [PySpark] - Add broadcast function by mariotaddeucci in 15037 feat: refactor getting tie_break_offset in SelectBestMatch by stephaniewang526 in 15235 Added dashes to test case csv_buffer_size_rejects.test_slow by hannes in 15398 [Dev] Split last part of ColumnDataCheckpointer::Checkpoint into FinalizeCheckpoint by Tishj in 15388 Fix JSON reader hang found by fuzzer by lnkuiper in 15397 Better partition selection for external hash joins by lnkuiper in 15389 fix arm extensions ci by samansmink in 15400 Feature 12699: ROW_NUMBER Secondary Sorts by hawkfish in 15403 Improve hash combining by lnkuiper in 15408 allow multifilereaders to delete entire chunks in FinalizeChunk by samansmink in 15401 Fix issue 14659 by pdet in 15411 Fix for issue 14648 by pdet in 15409 Re-enable some tests, removing mode skip or moving it later by carlopi in 15488 [Fix] Adjust reclaim space test to smaller block size nightly by taniabogatsch in 15414 Feature 12699: CUME_DIST Secondary Sorts by hawkfish in 15413 Fix issue with cleanup of buffers when reading same file multiple times by pdet in 15358 [Fix] Track correct allocation size of evicted memory by taniabogatsch in 15433 Fix internal issue 3813 by lnkuiper in 15427 Exploit RFC_4180 to be more strict with newline settings by pdet in 15426 Adds comment to Python Object + small adjustment do sniffer with comment detection. by pdet in 15425 Fix more nightly test errors due to sampling by Tmonster in 15423 Type mismatch set operation by Tmonster in 15422 Making the names option of CSV Files more restrictive when reading one file. by pdet in 15431 [Python][Dev] Lock mypy at 1.13 by Tishj in 15448 Fix InFilter::ToString, visible via EXPLAIN ANALYZE for example by carlopi in 15487 Mention configuration option that avoids total string size error in error message by soerenwolfers in 15489 Fix the seed of RandomLocalState to be 64bit instead of 32bits by carlopi in 15482 Fix ADBC Leak when reusing statements by pdet in 15475 chore: Add physical type translations for new timestamp types by krlmlr in 15472 [Dev] Slight cleanup of assert.hpp by Tishj in 15453 Retain join partition order by lnkuiper in 15460 Use system threads for parallelism on read_csv if reading from pipe by pdet in 15461 C API header generation for Go bindings by taniabogatsch in 14944 Move InitSegment into roaring namespace (nit) by arjenpdevries in 15495 chore: Add header for g++15 compatibility by krlmlr in 15509 Functions can throw errors by Tmonster in 15166 Improve candidate error message and relax constraint of rfc_4180 = false on quotes by pdet in 15371 Implement Union By Name on read csv relation by pdet in 15452 Add behaviour to remove unescaped quotes of unquoted values by pdet in 15454 [CSV Sniffer] If a column with Time/Date/Timestamp values encounter any other value, immediately go to VARCHAR by pdet in 15494 Introduce 2 new platforms: musllinux_arm64 and musllinux_amd64 by carlopi in 15429 15128: failed to bind column reference for function under unnest. by Tmonster in 15421 Setting descrialise Secondary Orderings by hawkfish in 15541 Throw IO exception on 1.1.3 database file with incorrect dependency order by taniabogatsch in 15568 Use ISNULL in conjunction or filters by Tmonster in 15529 Avoid fast fail: change defaults by tqml in 15474 Implicit STRUCT to STRUCT cast for mismatching member names by taniabogatsch in 15477 make test always fail in case of internal exception by c-herrewijn in 15569 CI: Bump container for Android build by szarnyasg in 15[Dev] ColumnDataCheckpointer can now checkpoint column data and validity data together by Tishj in 15566 Feature 12699: Secondary Sort Framing by hawkfish in 15523 [Test] More STRUCT cast tests by taniabogatsch in 15578 Making RFC4180=T DatabaseInstance's destructor: avoid throwing (and not cleaning up) by carlopi in 15707 Bugfixes by lnkuiper in 15704 Remove iceberg, again by carlopi in 15716 Allow shift-tab to be used to revert auto-complete suggestion, and implement SHOW [table] auto-completion by Mytherin in 15708 [Dev] Fix alignment issue in Roaring compression method by Tishj in 15711 Minor fixes by Mytherin in 15715 Move the DatabaseCacheEntry into the DBConfig, and set it before the constructor is called by Mytherin in 15714 Patching comparison operators in ICU to actually return bool by hannes in 15700 Preserve stack trace information when re-throwing by NiclasHaderer in 15709 [MultiFileReader] Extend support for column mapping from local -> global column by Tishj in 15446 Fix Arrow extension type Locks by pdet in 15705 Dont encode + on URL by pdet in 15693 Print an error when using "duckdb -f [file]" on a file that does not exist by Mytherin in 15718 Implement parquet_version parameter for Parquet writer by lnkuiper in 15684 [Testing] Temporarily skip tests by taniabogatsch in 15727 Add NATIVE_ARCH option to compile using -march=native, and in the CLI time queries that are send through "-c" by Mytherin in 15726 Remove httpfs patch by lnkuiper in 15729 Fix 15659: VARCHAR parameters now count as STRING_LITERAL again by Mytherin in 15724 Parquet reader: fix for filter on file_row_number column by Mytherin in 15736 Scan validity from dictionary vectors directly, and skip scanning validity when we encounter a dictionary vector by Mytherin in 15737 Make entries field non-nullable for Arrow map type by samansmink in 15733 Properly set external flag again in RadixPartitionedHashTable by lnkuiper in 15728 Storage version 65 by carlopi in 15702 Enable index scan for dynamic IN filter by taniabogatsch in 15665 Ignore pushes to version branches by Mytherin in 15743 Move changes in v1.2 to main by Mytherin in 15744 Initialize create_index_info.catalog by philippmd in 15738 Feature 15717: Window GROUPS by hawkfish in 15739 Fetch only required columns in physical delete by taniabogatsch in 15746 Add duckdb secret types function by samansmink in 15564 First round of extension bumps by Maxxen in 15655 Move core_functions to use unity builds by Mytherin in 15753 Add disabled_compression_methods setting that can be used to disable certain compression methods by Mytherin in 15754 Add support for deserializing a list of SetOperations in the SetOperationNode by Mytherin in 15755 Feature 15717: Window GROUPS by hawkfish in 15761 Check for mark join indexes in aggregate and group by by Tmonster in 15691 Default end of binding to varchar and not bool in CSV Reader by pdet in 15747 If arrow extension is not registered, use format information instead of failing by pdet in 15749 Merge 1.2 into main by Mytherin in 15769 Fix CI for Linux Release Building by hannes in 15748 Merge changes in main into v1.2 by Mytherin in 15770 When loading LogicalDependency from a database file or WAL file, modify the catalog to the catalog that we are loading into by Mytherin in 15767 Fix minor DuckDB-Wasm problem with stacktraces, that would be shown twice by carlopi in 15765 Move the instance cache entry when configuring by Mytherin in 15768 nitpick: Sequence Scan -> Sequential Scan by Mytherin in 15772 Bundle MingW static library with the default extension configuration by taniabogatsch in 15774 [Fix] Fix truncate + FK internal exception and another index bug by taniabogatsch in 15771 Switch logging to macros by samansmink in 15751 Add back Iceberg extension by carlopi in 15780 Internal 4002: SQLite EXCLUDE Tests by hawkfish in 15785 Skip 3 tests, to be reviewed on a side by carlopi in 15790 Add MD to autoload list by Mytherin in 15797 Connection manager: make count available without a lock by keeping track of it with an atomic by Mytherin in 15798 Add STORAGE_VERSION option that allows you to specify the target storage version when serializing a database by Mytherin in 15794 Fix some memory/storage issues in CI by lnkuiper in 15795 Fix map_extract backwards compatability by Maxxen in 15799 Fixes for vsize=2 tests by Mytherin in 15809 Fix tests for storage 65 by carlopi in 15807 Enable tests using no_alternative_verify by ywelsch in 15806 V1.2 histrionicus by Mytherin in 15812 Fix dependency conflict in PK FK benchmark by taniabogatsch in 15800 Remove shuffle from sampling by Tmonster in 15703 bump inet by Maxxen in 15804 Fix map_inference_threshold issue in JSON reader by lnkuiper in 15802 [CI] Invert operations for Linux CLI: first deploy, then test by carlopi in 15820 Fixup shell & autocomplete versioning information by carlopi in 15823 Skip end of test/sql/storage/parallel/insert_many_compressible_batches.test_slow by carlopi in 15814 Attempted parquet warning fix by Mytherin in 15827 Issue 15758: Streaming LEAD Buffering by hawkfish in 15834 Removing all core code and CI related to the substrait extension by pdet in 15810 CSV AFL Tests by pdet in 15805 improve error messages for mismatching versions of extensions by samansmink in 15829 dbgen: correctly join threads in case an error is thrown while generating data in parallel by Mytherin in 15840 Do not change type of empty files, if the types were manually set by pdet in 15841 Fix 15760 - when a SQL value function conflicts with an alias in the WHERE clause, prefer the alias by Mytherin in 15842 Fix 15570: preserve alias when using bind_replace in table functions by Mytherin in 15843 Fix CAPI chunk tests by pdet in 15846 fix: Fix compiler warning for uninitialized access by krlmlr in 15849 Relax RFC_4180=False a bit more flexible by pdet in 15832 More lenient test limits by Mytherin in 15845 bump delta, remove patches by samansmink in 15824 enable autoloading for iceberg and delta for storage by samansmink in 15822 Fix get_current_time, today, current_date backwards compatibility by Maxxen in 15803 Reset buffer before allocating a new one in ResizableBuffer by lnkuiper in 15838 V1.2 histrionicus by Mytherin in 15851 [tpch] dbgen: Avoid throwing interrupt that can't be caught by carlopi in 15856 Add CI run testing also slow tests on PRs by carlopi in 15854 More memory for external aggregate test by Mytherin in 15861 Fixes for nightly tests related to the CSV Parser by pdet in 15855 Fix latest storage tests CI by Mytherin in 15863 Fix duckdb_extensions() listing by carlopi in 15858 Use const T& and T instead of const T&& and T&& in (de)serializer by Mytherin in 15866 Make tests more lenient for smaller block sizes by Mytherin in 15872 Remove default in MultiFileReaderColumnDefinition constructor by Mytherin in 15871 Fix spurious test/sql/copy/partitioned/partitioned_write_tpch.test_slow:53 error by pdet in 15869 BindLogicalType should return a new type, instead of modifying an existing type in-place by Mytherin in 15868 V1.2 histrionicus by Mytherin in 15875 Issue 15877: CUME_DIST Moving Frame by hawkfish in 15878 Nightly CI fixes by Mytherin in 15885 Disable the RealNest benchmark nightly by hmeriann in 15839 disable iceberg tests by samansmink in 15883 [Linux CI] Remove examples, already tested as part of OSX Release by carlopi in 15879 Fix fuzzer issue found by the DuckFuzzer by pdet in 15886 Avoid unnecessarily reading the string dictionary size when scanning uncompressed strings by Mytherin in 15887 GCC-4.8 fixes by Mytherin in 15884 Several nightly CI fixes by Mytherin in 15889 Merge main into v1.2 by Mytherin in 15895 When Deserializing, Sample Selection Vectors should be initialized to FIXED_SAMPLE_SIZE by Tmonster in 15890 Faster re-builds by hannes in 15891 Add missing ExpressionType::COMPARE_NOTEQUAL no arrow pushdown by pdet in 15892 Fix race/deadlock in FixedSizebuffer::Get() by Maxxen in 15893 Call ProcessError also for PendingQueries by carlopi in 15899 Removed unused variable in LoggingContext by NiclasHaderer in 15898 CI: Handle 'fixed on nightly' label by szarnyasg in 15900 CheckMagicBytes: zero initialise buffer by carlopi in 15902 Rename RFC_4180 to STRICT_MODE. Change default to true. Use the same option in the sniffer as the parser. by pdet in 15896 Fix Arrow Type Registration on Extensions by pdet in 15901 V1.2 histrionicus by Mytherin in 15909 Use Arrow extension GetType() implementation when converting Arrow arrays by paleolimbot in 15813 @ text @d1 1 a1 1 # $NetBSD: Makefile,v 1.7 2025/01/01 10:22:38 ryoon Exp $ d3 1 a3 1 DISTNAME= duckdb-1.2.0 d20 1 a20 12 CMAKE_CONFIGURE_ARGS+= -DBUILD_CORE_FUNCTIONS_EXTENSION=ON CMAKE_CONFIGURE_ARGS+= -DBUILD_ICU_EXTENSION=ON CMAKE_CONFIGURE_ARGS+= -DBUILD_PARQUET_EXTENSION=ON CMAKE_CONFIGURE_ARGS+= -DBUILD_TPCH_EXTENSION=ON CMAKE_CONFIGURE_ARGS+= -DBUILD_TPCDS_EXTENSION=ON CMAKE_CONFIGURE_ARGS+= -DBUILD_FTS_EXTENSION=ON CMAKE_CONFIGURE_ARGS+= -DBUILD_HTTPFS_EXTENSION=ON CMAKE_CONFIGURE_ARGS+= -DBUILD_VISUALIZER_EXTENSION=ON CMAKE_CONFIGURE_ARGS+= -DBUILD_JSON_EXTENSION=ON CMAKE_CONFIGURE_ARGS+= -DBUILD_JEMALLOC_EXTENSION=ON CMAKE_CONFIGURE_ARGS+= -DBUILD_EXCEL_EXTENSION=ON CMAKE_CONFIGURE_ARGS+= -DBUILD_INET_EXTENSION=ON a21 1 CMAKE_CONFIGURE_ARGS+= -DBUILD_SQLSMITH_EXTENSION=ON a22 1 CMAKE_CONFIGURE_ARGS+= -DBUILD_AUTOCOMPLETE_EXTENSION=ON @ 1.7 log @databases/duckdb: Update to 1.1.3 Changelog: 1.1.3: This is a bug fix release for various issues discovered after we released 1.1.2. There are no new major features, just bug fixes. Database files created by DuckDB versions all the way back to v0.9.* can be read by this version. What's Changed * [Adaptive Sniffer] In case files have only one row, be more permissive to detect headers and types. by @@pdet in #14174 * Increase bounds for test/sql/copy/file_size_bytes.test by @@lnkuiper in # 14367 * Use table-level locking when acquiring shared locks by @@ywelsch in #14370 * [Arrow] Fix scanning of BOOL columns when offsets are involved by @@Tishj in #14395 * avoid unnecessary file list materialization when pruning readers by @@samansmink in #14397 * Fixing type pushdown on the CSV Scanner by @@pdet in #14399 * Issue #14398: Lead Shift Defaults by @@hawkfish in #14409 * Escape should default to quoted by @@pdet in #14418 * Sniff CSV rejects options and small sniffer fix for ignore_errors by @@pdet in #14417 * Fix #14430 - throw an error when reading corrupt statistics in the perfect hash aggregate by @@Mytherin in #14442 * Use corrent container to produce BinderErrors by @@carlopi in #14437 * Rework list_concat to accept a variable number of arguments similar to string concat by @@Mytherin in #14443 * SingleFileBlockManager::MarkBlockAsUsed - also erase from newly_freed_list to ensure trim does not prune blocks that are in-use by @@Mytherin in #14467 * acosh: Change example to avoid returning NaN by @@szarnyasg in #14477 * Don't move lvalue when inserting in order preserving map by @@Maxxen in # 14492 * bump vss by @@Maxxen in #14493 * Bug in percentage calculation in query_graph by @@bjornasm in #14494 * fix: standardize usage of LogicalType::ROW_TYPE for COLUMN_IDENTIFIER_ROW_ID by @@rustyconover in #14480 * [Python][Dev] Fix up test to work with older version by @@Tishj in #14505 * Avoid throwing on failure to open extensions's .info file (when force installing) by @@carlopi in #14272 * Python 3.7 tests are particularly brittle, we will keep building wheels but avoid testing them by @@carlopi in #14510 * Being more restrictive with the names option in the csv reader by @@pdet in #14466 * [Arrow] Fix issue where uninitialized memory was being read when scanning empty lists by @@Tishj in #14538 * Fix #14545 - pivot header must be defined in the grammar by @@Mytherin in # 14556 * [Python][Arrow] Cast to py::bytes when dealing with BLOB in filter pushdown by @@Tishj in #14553 * [Dev] Include aliases for RETURNING list expressions in ToString by @@Tishj in #14533 * CI: Mirror 'reproduced' / 'under review' to the internal repository by @@szarnyasg in #14527 * Approx-Top K: Make aggregate state trivially destructible by @@Mytherin in # 14571 * Don't pre-initialize hash vector in DistinctStatistics construction by @@abramk in #14570 * Fix query_graph tool for #14290 by @@JasonPunyon in #14521 * Fix typos by @@deining in #14579 * [Dev] Traverse the replace_list of StarExpression in ParsedExpressionIterator::EnumerateChildren by @@Tishj in #14535 * chore: Add EOL to source files by @@krlmlr in #14583 * Issue template: Add Swift redirect by @@szarnyasg in #14588 * Fix for underflow issue on number of rows in the CSV Reader by @@pdet in # 14587 * Cas strong by @@lnkuiper in #14592 * HTTPFS: HTTPException no longer inherits from IOException by @@Mytherin in # 14585 * [bufferpool] Fix PurgeAgedBlocksInternal() evictions by @@Vegetable26 in # 14446 * CI: Add bot for 'minimal reproducible example' by @@szarnyasg in #14598 * CI: Fix and simplify 'needs reproducible example' labelling by @@szarnyasg in #14608 * CI: Fix and simplify 'needs reproducible example' labelling by @@szarnyasg in #14609 * Enable serialization of LogicalExport by @@Mytherin in #14594 * Test fixes for new arrow release by @@pdet in #14593 * CI: Fix labelling bot by @@szarnyasg in #14611 * CI: Add repo name to labelling script by @@szarnyasg in #14612 * Unexpected result comparing blob by @@Tmonster in #14604 * Storage info update by @@szarnyasg in #14371 * Fix #14600: use UUID to generate unique pivot enum names by @@Mytherin in # 14622 * Fix #14601: avoid exporting entries in the temp or system schema by @@Mytherin in #14623 * Issue #14618: Year Day Year by @@hawkfish in #14624 * Fix #14542 by @@lnkuiper in #14610 * add index plan callback to IndexType by @@Maxxen in #14511 * FIX: Discrepancy Between Count and Sum Queries in SQL by @@Tmonster in # 14634 * Fix Windows Extensions CI by @@Mytherin in #14643 * chore: Add qualification for brotli code by @@krlmlr in #14628 * fix: Initialize atomic class member by @@krlmlr in #14627 * Fix secret serialization issues by @@samansmink in #14652 * Add serialization for bitstring_agg function by @@ywelsch in #14654 * Force error on CSV Sniffer Failure by @@lcostantino in #14661 * bump vss + spatial by @@Maxxen in #14667 * Bump extensions: AWS, Delta, Iceberg, INET by @@carlopi in #14669 * fix scoping problem with function argument by @@Damon07 in #14666 @ text @d1 1 a1 1 # $NetBSD: Makefile,v 1.6 2024/10/31 14:39:57 ryoon Exp $ d3 1 a3 1 DISTNAME= duckdb-1.1.3 @ 1.6 log @databasea/duckdb: Update to 1.1.2 Changelog: v1.1.2 Bugfix Release This is a bug fix release for various issues discovered after we released 1.1.1. There are no new major features, just bug fixes. Database files created by DuckDB versions all the way back to v0.9.* can be read by DuckDB v1.1.2. What's Changed * [CI] Re-enable ART zero initialisation verification by @@taniabogatsch in # 14031 * Push filters instead of overwriting filters by @@Tmonster in #14078 * Fix test by @@Tmonster in #14079 * fix maximum_threads test inside containers by @@wenjun93 in #14083 * Fix: remove is_probe_in_domain by @@Light-City in #14084 * Add duckdb_extension.h to amalgamation release by @@samansmink in #14086 * Bump minimum required cmake version by @@abramk in #14089 * Fix parser error by removing alias by @@taniabogatsch in #14090 * [Dev] Move EnumTypeInfoTemplated definition into a hpp file by @@Tishj in # 14038 * Fix #14077: correctly reset next pointer when reconstructing new row group segment tree after vacuum by @@Mytherin in #14092 * Format CSV error messages by @@szarnyasg in #14097 * Fix the answer file for tpcds q67 at sf100 by @@Tmonster in #14096 * Add v1.1.1 to version_map.json by @@Flogex in #14110 * CREATE TABLE now supports columns with ENUM[] types. by @@Tishj in #14102 * fix parquet cardinality when first file is empty by @@wenjun93 in #14058 * [Python Dev] Make sure the GIL is released when the connection+db are being shut down by @@Tishj in #14113 * Less salt by @@lnkuiper in #14173 * remove redundant code by @@flashmouse in #14172 * comparison of nested types returns true or false always (even with nulls) by @@Tmonster in #14094 * subtype DBInterface.Connection in julia client by @@piever in #14193 * [Python] Fix a bug with python_scan_all_frames reaching the bottom of the frame stack by @@Tishj in #14183 * [Dev] Fix issue where the InsertStatement::ToString call destroyed the alias of the ValuesList by @@Tishj in #14171 * [Python] Fix issue related to scanning float64 dtype columns that contain a mask by @@Tishj in #14170 * Fix some warnings found while compiling duckdb-node by @@carlopi in #13994 * fix minmax type info miss by @@Damon07 in #14159 * fix: ArrowSchemaMetadata::GetOption to return empty string instead of raising exception if key is not found. by @@rustyconover in #14157 * Issue 14151: Fix conflicting defines on Windows hidden by cmake unity builds by @@zmajeed in #14154 * Issue 14189: Fix build when threads are disabled by @@zmajeed in #14190 * Fix an uncaught error with a generated column containing a subquery by @@Tishj in #14198 * Add missing word in TableFunction comment by @@JelteF in #14210 * add method to check whether julia connection is open by @@piever in #14195 * Avoid schema changes with IF NOT EXISTS by @@ywelsch in #14143 * Fix typos in code by @@c8ef in #14243 * [Dev] Add the ExecutorException class, making use of the EXECUTOR ExceptionType by @@Tishj in #14231 * [Python] Don't allow construction of DuckDBPyType from empty Dict type by @@Tishj in #14221 * Fix #14232: fix deliminator optimizer by @@flashmouse in #14238 * [CSV Reader] Also use figure-out-line code when ignoring errors. by @@pdet in #14184 * remove redundant Bit::SetBit by @@flashmouse in #14226 * Fix #14212: mention correct query component when using literal in DISTINCT ON by @@Mytherin in #14255 * Removing overzealous check in Parquet by @@hannes in #14268 * Update sqlsmith extension and patches by @@Tmonster in #14270 * Support for duckdb.varint extension in Arrow. by @@pdet in #14273 * Disable CSV ignore_errors benchmark by @@pdet in #14277 * Only slice initialized vectors in PhysicalHashAggregate::SinkDistinctGrouping by @@lnkuiper in #14289 * Fix #14249: return NAN when dividened is 0 by @@flashmouse in #14298 * Revert "Fix #14249: return NAN when dividened is 0" by @@Mytherin in #14308 * fix macro name with the same function name in it which causing repeat?? by @@Damon07 in #14296 * Fixing issue with the sniffer on copy statetements by @@pdet in #14295 * Json bugfixes by @@lnkuiper in #14288 * [Bitstring] Add overload for bitstring to accept BIT as the type of the first argument by @@Tishj in #14247 * [Fix] Don't initialize reference, constant, and parameter children in intermediate chunk by @@taniabogatsch in #14254 * Profiling - correct settings per node type and minor renaming for clarity by @@taniabogatsch in #14290 * Fix extension size increase by @@samansmink in #14185 * Add option to ignore GeoParquet, disable spatial autoloading when reading GeoParquet by @@Maxxen in #14297 * More defensive programming in RowVersionManager::CleanupAppend by @@Mytherin in #14317 * Change Makefile to correctly handle DISABLE_SANITIZER and DISABLE_UBSAN by @@JelteF in #14316 * [CSV Reader] Making escape not limited to only quotes by @@pdet in #14314 * allow external cardinality information (e.g. from iceberg) by @@peterboncz in #14292 * [SecretManager] Fix deserialization of Value types in KeyValueSecret::Deserialize by @@Tishj in #14332 * Avoid throwing InternalException on reading secret by @@carlopi in #14336 * delay the rewrite of a large IN-clause into a MarkJoin on remote Filter-Scans by @@peterboncz in #14266 * JSON reader - never generate maps if map_inference_threshold is -1 by @@Mytherin in #14348 * [Appender] Support appending to table with generated columns by @@Tishj in # 14346 * Internal #3251: DateDiff Across Epoch by @@hawkfish in #14338 * Bump azure and delta extensions commits by @@carlopi in #14350 * Bump spatial to 3f94d52aa9f7d67b1a30e6cea642bbb790c04aa2 by @@carlopi in # 14351 * Bump more extensions: iceberg, vss and sqlite_scanner by @@carlopi in #14352 * Emit profiling info for extension operators by @@Mytherin in #14355 v1.1.1 Bugfix Release This is a bug fix release for various issues discovered after we released 1.1.0. There are no new major features, just bug fixes. Database files created by DuckDB since v0.9.* can be read by DuckDB v1.1.1. What's Changed * [Python] Fix a crash related to handling of the f_locals by @@Tishj in # 13821 * [Dev] Remove unnecessary parameter from BufferHandle constructor by @@Tishj in #13823 * When vacuuming, immediately checkpoint the vacuumed row groups instead of scheduling a checkpoint task by @@Mytherin in #13825 * Check for unquoted errors after finishing up a buffer. by @@pdet in #13826 * Issue #13813: TIMETZ Uninvertible Casts by @@hawkfish in #13820 * add cardinality for cross product and propositional join by @@jeewonhh in # 13818 * Add requested_schema argument to PyCapsule interface by @@WillAyd in #13802 * Throw at double/float cast to decimal if it does not fit by @@pdet in #13745 * [Fix] C API - Correct type comparison in MAP value functions by @@taniabogatsch in #13844 * [C-API Dev] Fix up "Test DataChunk populate ListVector in C API" test by @@Tishj in #13839 * [Dev] Minor cleanup to BufferManager and BlockHandle API by @@Tishj in # 13838 * Small fixes for prompt of sniff_csv by @@pdet in #13843 * [BufferManager] Fix duckdb_memory() reporting wrong size for temporary_storage_bytes by @@Tishj in #13837 * Fix for internal error when using rejects tables and adding implicit cast for boolean values. by @@pdet in #13849 * [Fix ART] Correct prefix transformation by @@taniabogatsch in #13858 * [Python] Fix issue causing an exception when creating a duckdb.StarExpression without an exclude_list by @@Tishj in #13881 * Fix issue with cgroups/slurm variables: skip if memory limit cannot be parsed, and only run this on Linux by @@Mytherin in #13879 * Explicit windows-2019 instead of windows-latest by @@carlopi in #13883 * CI/Windows: Drop redundant package by @@szarnyasg in #13874 * Preserve operator in BindWithCTE by @@lnkuiper in #13873 * Update description of 'max_temp_directory_size' by @@szarnyasg in #13724 * 13810 unnest cross join error by @@Tmonster in #13878 * Tweak allocation purging by @@lnkuiper in #13877 * Copy From Database - create a balanced UNION ALL tree instead of a depth-first union all tree to avoid stack overflows by @@Mytherin in #13889 * [Python] Fix issue related to the GIL when using execute with multiple statements by @@Tishj in #13893 * Fix #13880: correctly name http_proxy_password setting by @@Mytherin in # 13890 * Fix #13872: duckdb_result_return_type is not deprecated, and group together deprecated functions by @@Mytherin in #13891 * [Python] Add python_scan_all_frames to opt-in to scanning all frames (< 1.1 behavior) by @@Tishj in #13896 * Improve error on enums by @@carlopi in #13885 * Handle extension ABI mismatches in a forward & backward compatible way by @@carlopi in #13894 * Fix #13824: min() max() varchar column use default collation by @@flashmouse in #13909 * Fix issue in casting 2 byte BIT -> BLOB by @@Tishj in #13908 * add missing azure secret providers for autoloading by @@samansmink in #13902 * Remove buffer_manager_allocate.patch and bump spatial by @@carlopi in #13895 * [Python] Improve install_extension to support repository/repository_url and version by @@Tishj in #13876 * fix REGEX not supported anymore for valid queries (only statement error) # 2889 by @@hmeriann in #13633 * [CI] Invert operations on OSX.yml, deploying nightly artifacts before test by @@carlopi in #13927 * [Python][Jupyter] Don't use ExplainFormat::HTML for explain('analyze') by @@Tishj in #13928 * Only bind to SQL value functions if there is no alias with this name present we can bind to instead by @@Mytherin in #13925 * Improve logic for remote extension install on Windows by @@carlopi in #13929 * CI: Trigger actions for labeled discussions by @@szarnyasg in #13937 * [Swift] Update README.md in Swift repo by @@tcldr in #13955 * [Dev] Add exclusion for pybind11 internal _pybind11_conduit_v1_ method by @@Tishj in #13961 * Create a balanced union tree, also for export by @@Mytherin in #13956 * Increment julia version by @@Mytherin in #13966 * Fix #13585 - transform from or select first based on order specified by the user by @@Mytherin in #13959 * Fix Cross Product Cardinality by @@jeewonhh in #13954 * Do not run the date/timestamp format sniffer if they are set by the user by @@pdet in #13960 * Modify create_art_varchar.benchmark so it passes weekly regressions by @@Tmonster in #13968 * Fix data race when writing GeoParquet by @@Maxxen in #13962 * Check vector type in GetVectorScanType to avoid concurrent race when updating validity by @@Mytherin in #13965 * Parser Keyword Category Search by @@pdet in #13875 * Escape quotes in FTS by @@lnkuiper in #13917 * Fix #13941: fix error message in appender by @@Mytherin in #13957 * fix: remove http prefix from proxy value when present by @@dylanspag-lmco in #13973 * Use defaults when serializing copy to file by @@Mytherin in #13978 * Fix #13933: disable join filter pushdown when a join is performed over collated columns by @@Mytherin in #13958 * Fix partitions on wide tables by @@piever in #13988 * [Fix] Throw exception for UNNEST in lambdas by @@taniabogatsch in #13969 * Fixing some parquet issues found by fuzzing by @@hannes in #13964 * fix julia partitions for streaming result by @@piever in #14000 * More descriptive Parquet created_by with version and source hash by @@hannes in #13992 * Decimal downcast limit check by @@pdet in #13922 * [C API] Add SQLNULL to the duckdb types by @@taniabogatsch in #13999 * Fix crash in the shell caused by printing blobs that failed to cast by @@Maxxen in #13983 * [Binding] Always try binding with the schema of the UserType first if it's set by @@Tishj in #13995 * [Arrow] Only produce 'arrow.json' Extension types when arrow_lossless_conversion is enabled. by @@Tishj in #13989 * Bugfixes by @@lnkuiper in #13982 * Disable swift linux tests by @@Mytherin in #14019 * fix minor typos in comments of aggregate function tests by @@era127 in # 14007 * [CSV Sniffer] Slight change of rules for dialect detection by @@pdet in # 14013 * [Test Fix] Add noforcestorage to in-memory tests by @@taniabogatsch in # 14016 * Fix #14020: fix off-by-one in RLE compression: avoid flushing when last_seen_count == 0 which can happen if a column has exactly 2^16 (65535) repeated values by @@Mytherin in #14030 * Adds Julia support for scalar UDFs by @@drizk1 in #14024 * Proper NULL handling in special json extraction functions by @@lnkuiper in # 14032 * Fix #13993 - avoid disabling optimizers for SET VARIABLE by @@Mytherin in # 14028 * [Arrow] Make unknown Arrow extensions throw at scan instead of bind by @@pdet in #14015 * Improve assertion macros by @@c-herrewijn in #14033 * [Arrow] Move ArrowUtil to its own file by @@Tishj in #13990 * [CSV Sniffer] Verify validity of header before value access by @@pdet in # 14040 * Fix #14026: all TIMESTAMP_xx cannot cast to TIME directly by @@flashmouse in #14045 * Only merge distinct stats if both sides have distinct stats available by @@Mytherin in #14046 * Avoid http-redirect in README by @@carlopi in #14029 * fix: make ArrowArrayWrapper::GetNextChunk() virtual by @@rustyconover in # 14003 * Issue #13655: MEDIAN Even Tests by @@hawkfish in #13722 * Fix #13934: use CreateSortKeyWithValidity to correctly handle NULL in all calls to arg_max by @@Mytherin in #14053 * Upgrade MySQL/Postgres extensions by @@Mytherin in #14054 * [Union Reader] Early-out on readers of files that do not have data by @@pdet in #14050 * Issue #13899: AsOf Unrelated Pushdown by @@hawkfish in #13975 * Feature #3128: 2024b Time Zones by @@hawkfish in #14061 * bump spatial by @@Maxxen in #14062 * Fixup StagedUpload invocation via workflow_call by @@carlopi in #14065 * Skip polars test in 3.7 due to missing PanicException by @@carlopi in #14066 * Add building, testing and distributing for Python 3.13 by @@carlopi in # 14067 * bump sqlsmith and azure versions by @@samansmink in #14069 * bump substrait by @@pdet in #14055 * CIBW_SKIP also musllinux on Python 3.7 or Python 3.8 by @@carlopi in #14074 @ text @d1 1 a1 1 # $NetBSD: Makefile,v 1.5 2024/09/17 06:10:43 ryoon Exp $ d3 1 a3 1 DISTNAME= duckdb-1.1.2 @ 1.5 log @databases/duckdb: Update to 1.1.0 Changelog: 1.1.0 What's Changed * Add feature changes back in by @@Mytherin in #11146 * Make MultiFileReader filename configurable by @@lnkuiper in #11178 * [Dev] Fix compilation issues on feature by @@Tishj in #11082 * add query() and query_table() functions by @@chrisiou in #10586 * [Block Size] Move the block allocation size into the block manager by @@taniabogatsch in #11176 * LIMIT pushdown below PROJECT by @@jeewonhh in #11112 * BUGFIX: IN () filter with one argument should translate to = filter. by @@Tmonster in #11473 * Regression Script should calculate micro benchmark differences with the correct base branch by @@Tmonster in #11762 * Pushdown filters on window partitions by @@Tmonster in #10932 * Arrow ListView Type by @@Tishj in #10766 * Add scalar function support to the C API by @@Mytherin in #11786 * Add TopN optimization in physical plan mapping by @@kryonix in #11290 * Join-dependent filter derivation by @@lnkuiper in #11272 * Implement ROW_GROUPS_PER_FILE for Parquet by @@lnkuiper in #11249 * Prefer Final projected columns on probe side if cardinalities are similar by @@Tmonster in #11109 * Propagate unused columns to distinct on by @@Tmonster in #11006 * Separate eviction queues by FileBufferType by @@lnkuiper in #11417 * Disable false positive for vector size nightly in test by @@taniabogatsch in #11953 * Rework jemalloc extension by @@lnkuiper in #11891 * Tweak jemalloc config by @@lnkuiper in #12034 * Httpfs test to nightly by @@carlopi in #12196 * Removed three reinterpret casts and some rewriting by @@taniabogatsch in # 12200 * Begin Profiling Rework to move towards Modularity by @@maiadegraaf in #11101 * [CLI] Add highlighting + limited auto-complete for shell dot commands by @@Mytherin in #12201 * Skip test to fix block size nightly and add more explicit error checking by @@taniabogatsch in #12211 * Remove BLOCK_ALLOC_SIZE from the column segment files by @@taniabogatsch in #11474 * [Julia] - Added optional schema input argument to DuckDB.Appender constructor by @@curtd in #12174 * Fix Mark Index in the Bound Join Ref by @@pdet in #12263 * Fix for CI Regression Failure by @@maiadegraaf in #12273 * ? by @@samansmink in #12303 * Disable JEMALLOC_RETAIN by @@lnkuiper in #12185 * Enforce compression extensions for CSV Files by @@pdet in #11903 * Make spuriously failing test more robust by @@lnkuiper in #12306 * Add new extensions to issue template by @@szarnyasg in #12313 * [Fix] Block size nightly run by @@taniabogatsch in #12283 * Spell Check | Nothing Major | Corrected base_scanner.cpp by @@nj7 in #12282 * add duckdb_bind_timestamp_tz function to C API by @@karlseguin in #12151 * [Python] Add some date/datetime functions to pyspark api by @@mariotaddeucci in #12075 * Fixes to Windows workflow and ubuntu_18 action by @@carlopi in #12308 * [Extension Dev] Forward declare re2 in hive_partitioning.hpp by @@Tishj in # 12302 * add expected errors to test/sql/copy/per_thread_output.test by @@hmeriann in #12280 * Issue #12287: ICU Strptime Lists by @@hawkfish in #12295 * Issue #12171: Streaming Window FILTER by @@hawkfish in #12250 * [Python] Update the Connection wrapper generation, now generates c++ code by @@Tishj in #12216 * Use iterator buffer position when storing buffer handles by @@pdet in #12315 * Bump Julia client to v0.10.3 by @@Mytherin in #12323 * Fix #12286 - in the MetadataManager, prefer to allocate new blocks if the next free block id is smaller than the currently used metadata block by @@Mytherin in #12318 * [Fix] Only read file size if file handle still exists by @@taniabogatsch in #12319 * Add support for APPEND argument to hive partitioned write by @@Mytherin in # 12262 * Remove all reinterpret casts from the transformer by @@taniabogatsch in # 12320 * Additional check for overlapping CTE names by @@lnkuiper in #12305 * [Dev] STANDARD_VECTOR_SIZE and BLOCK_ALLOC_SIZE can now be set through the Makefile by @@Tishj in #12164 * [Upsert] Fix issue with lambdas in DO UPDATE SET expressions by @@Tishj in # 11866 * [Python] Fix scoping issue for pandas_analyze_sample setting by @@Tishj in # 11706 * Support REGEX matches expected error message by @@hmeriann in #12327 * Allow run_fuzzer to reduce multi statements. by @@Tmonster in #12278 * Fix #12328 - when flattening STRUCT vectors with NULL values, we need to flatten the children recursively as well by @@Mytherin in #12332 * Make dbgen generate data in parallel by @@Mytherin in #12337 * dbgen: skip parallel generation if DUCKDB_NO_THREADS is set by @@Mytherin in #12341 * Add prefix prefix_front_back. to get prefix_front_ and prefix_back_ by @@liujiayi771 in #12344 * Issue #12171: Streaming Windowed DISTINCT by @@hawkfish in #12311 * Update README by @@szarnyasg in #12357 * [CSV Reader] [Skip Option] Tests and fixes by @@pdet in #12213 * Adjust BM25 score in FTS extension to prevent negative scores by @@lnkuiper in #12356 * Fix typos by @@szarnyasg in #12360 * Fix #12293 - accept NULL values in generate_series with timestamp by @@Mytherin in #12367 * Fix #12335: avoid calling fsync when writing Parquet files, instead just close the file by @@Mytherin in #12371 * Fix parameters passed down to other workflows in OnTag.yml by @@carlopi in # 12369 * [Python] Fixes for the SQLLogicTest runner implementation by @@Tishj in # 12372 * Bump julia to v1.0.0 by @@Mytherin in #12379 * Fix #11921 - varchar -> timestamp casts are not invertible by @@Mytherin in #12376 * Upgrade utf8proc - and move our custom extensions out of utf8proc itself by @@Mytherin in #12373 * change max_queries number back to 2000 by @@Tmonster in #12375 * Remove sqlsmith extension by @@Tmonster in #12300 * Reorder semi and anti joins. by @@Tmonster in #11815 * Issue #12351: implicit cast to TIMESTAMP_MS, TIMESTAMP_S, TIMESTAMP_NS from DATE values by @@akoshchiy in #12352 * Issue #10023: Approx_Count_Distinct Memory Usage by @@hawkfish in #12355 * Fix a small typo in dev instructions for swift setup by @@gjmwoods in #12383 * Release lock before returning BufferHandle in StandardBufferManager::Pin by @@lnkuiper in #12391 * Remote attach autoload by @@carlopi in #12393 * Add JSON type to Parquet reader/writer by @@lnkuiper in #12222 * Add RETURN_FILES parameter to COPY TO by @@lnkuiper in #12220 * Updated JoinHashTable to use linear probing to resolve hash collisions by @@gropaul in #11472 * [Benchmark Runner] Add --disable-timeout flag by @@Tishj in #12387 * Don't replace unicode spaces within $$ quotes in query strings by @@lnkuiper in #12405 * [Python] Fix fatal exception caused by empty Pandas Categorical objects. by @@Tishj in #12370 * Release CSV Blocks when acquiring new blocks if single threaded by @@pdet in #12409 * Add support for prefetching multiple adjacent blocks in a single batched read when attaching to remote databases by @@Mytherin in #12413 * MatchRegex() fixed to do not return false positive result by @@hmeriann in # 12396 * Expected errors 2053 by @@hmeriann in #12392 * [C-API] Catch exception in duckdb_execute_prepared by @@Tishj in #12414 * Combining LIST_CONCAT and CONCAT binding by @@maiadegraaf in #12317 * [Appender] Add AppendDefault by @@Tishj in #11905 * [Python Dev] Push CTE internally for every (python) replacement scan that occurred. by @@Tishj in #12161 * Improve compiler compatibility by @@krlmlr in #12401 * Write zero-length list offsets for NULL values when serializing vectors by @@Mytherin in #12423 * Get column statistics if Logical Get has a statistics function by @@jeewonhh in #12424 * jemalloc: Identify GNU source code properly by @@lnkuiper in #12420 * Avoid parallelizing LIMIT clauses when the query plan is simple by @@Mytherin in #12433 * Prefetch metadata blocks for remote files by @@Mytherin in #12437 * [Jupyter] Remove width limit on the BoxRenderer config by @@Tishj in #12443 * Revert #10865 by @@carlopi in #12426 * inline delta by @@samansmink in #12435 * Account for tagged dollar-quoted strings when stripping unicode spaces by @@lnkuiper in #12421 * Work-around for broken github windows runner by @@Mytherin in #12447 * Prevents clearing of the types of the LogicalExecute operator by @@NiclasHaderer in #12436 * Add support for BEGIN TRANSACTION READ ONLY by @@Mytherin in #12202 * Make range and generate_series table in-out functions, and fix several issues with table in-out functions by @@Mytherin in #12431 * Issue #12412: AsOf Filter Push by @@hawkfish in #12448 * [Fix] Block Size Nightly by @@taniabogatsch in #12427 * [ART] Remove Flatten and template key generation by @@taniabogatsch in # 12428 * [Python] Clean up internals of execute / executemany by @@Tishj in #12434 * By default attach remote databases as READ_ONLY by @@carlopi in #12461 * Fix #11837: use internal physical type for FIRST/LAST/ANY_VALUE instead of logical type by @@Mytherin in #12462 * Issue #12464: Windowed Order By All by @@hawkfish in #12470 * Specialize list_value for primitive types for significantly improved performance by @@Mytherin in #12468 * [Dev] Remove dead code from PhysicalBatchCopyToFile by @@Tishj in #12459 * Disable Windows extensions CI until Github actions runners are fixed by @@Mytherin in #12479 * [Fix] access_mode now lives in AttachOptions by @@taniabogatsch in #12482 * Internal #2186: Nanosecond Functionality by @@hawkfish in #12440 * [C-API] Fix leak in duckdb_create_config by @@Tishj in #12465 * [Python] No longer scan the entire frame lineage in a replacement scan, added option to disable (python) replacements entirely by @@Tishj in #12425 * throw binder error for comment on system catalog by @@samansmink in #12486 * Parquet reader performance by @@lnkuiper in #12478 * Operators the Optimizer can skip by @@Tmonster in #12489 * Fixes clang conversion warnings by @@TinyTinni in #12467 * Avoid creating internal schemas as non-internal when reading old database files by @@Mytherin in #12456 * Allow parquet encryption/decryption keys to be passed in as base64 encoded strings by @@elefeint in #12445 * [Block Size] Introducing CompressionInfo by @@taniabogatsch in #12481 * add the number of filtered files to explain by @@samansmink in #12488 * Implement Map Type Detection for JSON Reader by @@ZiyaZa in #11285 * [Dev] Remove busy-spin from ClientContext::ExecuteTaskInternal by @@Tishj in #12483 * Pluggable collations by @@Mytherin in #12492 * [Dev] Don't fail make generate-files if the python code generation fails by @@Tishj in #12500 * Optimize EXTRACT(year/month/day FROM date/timestamp) by @@Mytherin in #12499 * [Fix] Remove BLOCK_ALLOC_SIZE in the single file block manager by @@taniabogatsch in #12502 * Revert Windows CI fixes by @@carlopi in #12510 * Fix #12467 changes to covariance calculation by @@carlopi in #12515 * [Python] Fix reading strided datetime and timedelta columns by @@Tishj in # 12519 * Add method for decoding sort keys, and use this in min/max for arbitrary types by @@Mytherin in #12520 * Reduce allocations & use predication in ColumnSegment::FilterSelection by @@Mytherin in #12521 * Skip only built-in optimizers by @@Mytherin in #12522 * Improve min/max performance for strings and fallback types by @@Mytherin in #12524 * Move arg_min/arg_max to use sort keys by @@Mytherin in #12525 * Move FIRST/LAST/ANY_VALUE to use sort keys by @@Mytherin in #12526 * CMake: use GNUInstallDirs as defaults for INSTALL_{BIN,LIB,INCLUDE}_DIR by @@paparodeo in #12509 * More formatting and fix to stddev by @@carlopi in #12516 * Linux Extensions CI: Attempt at fix missing dependencies by @@carlopi in # 12429 * Fix checkouts by @@carlopi in #12366 * Etag if none match for extension install by @@carlopi in #12333 * [Block Size] FixedSizeAllocator, MetadataManager, PartialBlockManager by @@taniabogatsch in #12514 * [Python] Skip the PandasAnalyzer if dtype is 'string' by @@Tishj in #12511 * [StreamQueryResult] Batched variant of the StreamQueryResult collector by @@Tishj in #11494 * Move many tests to slow by @@Mytherin in #12534 * Add support for arg_min(ANY, ANY) by @@Mytherin in #12532 * Avoid overriding types in PrepareTypeForCast when not required by @@Mytherin in #12539 * Support all types in histogram function by @@Mytherin in #12538 * [Python] Remove busy-spin during execution by @@Tishj in #12512 * [Block Size] String space constant by @@taniabogatsch in #12537 * Use string_t instead of std::string in histogram by @@Mytherin in #12545 * Add support for binned histograms by @@Mytherin in #12548 * [Upsert] Fix RETURNING for DO NOTHING by @@Tishj in #12554 * Build Android Binaries by @@hannes in #12550 * [CI] Remove pyarrow version lock by @@Tishj in #12566 * [Dev] Change tests: np.NaN -> np.nan by @@Tishj in #12565 * Internal #2017: DECIMAL Downcast Rounding by @@hawkfish in #12036 * Issue #12204: Summarize Temporal Quantiles by @@hawkfish in #12297 * Internal #2186: Nanosecond StrTimeFormat by @@hawkfish in #12551 * Add support for equi_width_bins function to compute histogram boundaries by @@Mytherin in #12574 * add support for casting 'yes'/'no' strings to boolean values by @@chrisiou in #12501 * Julia: Add chunked results with Tables.partitions() by @@frankier in #12395 * [PySpark] - Allow spark session range by @@mariotaddeucci in #12346 * [PySpark] Implement subset drop duplicates by @@mariotaddeucci in #12348 * ICU noaccent collation by @@tiagokepe in #12170 * Implement Brotli compression for Parquet reading & writing by @@hannes in # 12103 * [FriendlySQL] Unpacked COLUMNS() Expression by @@Tishj in #11872 * [PySpark] Implement UDFRegistration.register method on PySpark api by @@mariotaddeucci in #12179 * [Python] Don't use np.nan, deprecated alias starting with NumPy 2.0 by @@Tishj in #12583 * Add bind_expression callback to scalar function, and use it to turn typeof into a BoundConstantExpression by @@Mytherin in #12580 * Add can_cast_implicitly scalar function by @@Mytherin in #12581 * Add support for histogram and histogram_values table macro, and add support for default table macros (similar to how we support default macros) by @@Mytherin in #12590 * build: swap libclang for cxxheaderparser by @@Mause in #12567 * [C-API] Add table_description struct to query various information about the table. by @@Tishj in #12460 * Change new micro benchmark script to only look for .benchmark files by @@maiadegraaf in #12598 * Add HTTP error code to extension install failures by @@carlopi in #12608 * Separate WAL write from commit, and allow writing to the WAL without holding the transaction lock by @@Mytherin in #12261 * Add OwningStringMap - and rework histogram and mode functions to use this by @@Mytherin in #12601 * Feature #1272: Window Executor State by @@hawkfish in #12573 * Add support for any type to mode aggregate by @@Mytherin in #12619 * WAL - when dropping a table, also delete any transaction local storage associated with that table by @@Mytherin in #12603 * [Python] Allow Generators to be passed where List is expected by @@Tishj in #12602 * VectorOperations::Copy - fast path when copying an aligned flat validity mask into a flat vector by @@Mytherin in #12618 * Move android CI to only run during nightly CI triggers by @@Mytherin in # 12622 * Add initial support for GeoParquet + Bump spatial by @@Maxxen in #12503 * Issue #12600: Streaming Positive LAG by @@hawkfish in #12609 * Feature #1272: Window Group Preparation by @@hawkfish in #12628 * Minor window improvements by @@Mytherin in #12617 * Merge feature into main by @@Mytherin in #12633 * Refactor quantile aggregate - clean up code & support quantile_disc/median for all types by @@Mytherin in #12630 * Feature 1272: Window Payload Preallocation by @@hawkfish in #12629 * [ART] Configurable index scan threshold by @@taniabogatsch in #12635 * Subtract start offset for when fetching array child segment by @@Maxxen in # 12639 * Remove custom logic to detect main vs feature by @@carlopi in #12643 * Do not quote fields with space in the CSV output mode by @@szarnyasg in # 12644 * Use lowercase in 'html' output mode by @@szarnyasg in #12612 * Internal #2361: Window ROWS Overflow by @@hawkfish in #12652 * Quantile: Fix variable used only in D_ASSERT by @@carlopi in #12642 * Skip pytorch test, it fails spuriously in CI by @@carlopi in #12645 * Add histogram_exact function that adds values to bins only if they match exactly, and add other column that contains values that do not fit in any bin by @@Mytherin in #12650 * Add operator hook for sink progress by @@Maxxen in #12637 * Regression workflow on newly introduced benchmarks: remove for now by @@carlopi in #12659 * Fix #12646 - allow SQL value functions in HAVING by @@Mytherin in #12654 * Add != operators on string_t and interval_t by @@carlopi in #12658 * fix: improve C scalar functions API by @@rustyconover in #12663 * Add approx_top_k aggregate based on the (Filtered) Space-Saving algorithm, and use it in histogram by @@Mytherin in #12653 * Fix std::sort requirements, from greater_equal to greater by @@carlopi in # 12669 * fix(parquet): two-complement zeroes check on FIXED_BYTE_ARRAY encoded DECIMAL (#12621) by @@fedefrancescon in #12655 * [CSV Reader] Reorder of Columns for CSV Scans on multiple files. by @@pdet in #12288 * [CSV] [Bug-Fix] Fix for issue related with single-threaded execution and null padding. by @@pdet in #12679 * [Block Size] String block limit and a few other places by @@taniabogatsch in #12671 * Rework arena allocator allocation policy - and increase pivot threshold by @@Mytherin in #12690 * Julia - Fix Base.isopen(db::DB) in #12700 * [CLI] Limit history size to 100MB, and avoid writing invalid UTF8 to the CLI history by @@Mytherin in #12677 * Add configurable thresholds for using nested loop join and merge join by @@Mytherin in #12689 * Prevent unnecessary usage of std::string in list aggregate - and use more efficient memcpy for batched copy by @@Mytherin in #12694 * Dont load spatial unless geoparquet metata is present by @@Maxxen in #12692 * Serialization: add CustomData and better support for integrating with extensions by @@jeewonhh in #12681 * Removing ODBC driver by @@hannes in #12706 * Support thousand separator for floating point numbers by @@Mytherin in # 12717 * [Python] Use non-owning references to hold created cursors by @@Tishj in # 12711 * LIST(VARCHAR) - reduce memory usage by avoiding allocation of nullmask for string data, and allocate larger initial batches by @@Mytherin in #12705 * [CSV] Bug fix for race condition in single-threaded multifile reader + properly print paths on union_by_name errors. by @@pdet in #12697 * Issue template: Add ODBC and Node (neo) clients by @@szarnyasg in #12714 * Shell: add .sql suffix to temporary file created with \e by @@Mytherin in # 12723 * Partitioned write - keep only up until 100 files open, when this limit is exceeded close the file and create a new file if more data for this partition appears by @@Mytherin in #12708 * Change setting types to fix warnings by @@Mytherin in #12724 * Avoid unnecessarily copying child expression when binding COLLATE statements by @@Mytherin in #12725 * Support for variadic arguments in scalar UDFs in the C API by @@taniabogatsch in #12678 * [Relation API] Dont push DISTINCT modifier for EXCEPT/INTERSECT ALL by @@Tishj in #12599 * Builds for Windows on ARM64 by @@hannes in #12586 * Rework union_by_name so that files are no longer kept open by @@Mytherin in #12730 * Fix #12729: early-out when checking for perfect hash joins when running on empty tables by @@Mytherin in #12731 * CLI: Replace \n with \r\n again in history again by @@Mytherin in #12735 * Fix #11228 - add support for unsigned integers in printf/format by @@Mytherin in #12736 * Various CI fixes by @@Mytherin in #12737 * Add repeat(LIST[], INT) that allows repetition of lists similar to how this is allowed in Python by @@Mytherin in #12738 * [Python] Add missing options to read_json method by @@Tishj in #12732 * Add support for fetching cardinality estimation and stats through a multifilelist by @@samansmink in #12740 * Fixes warnings detected by cppcheck by @@carlopi in #12745 * [Arrow] Add ArrowQueryResult by @@Tishj in #12496 * [Dev] StreamQueryResult internals cleanup by @@Tishj in #12636 * ALP/ALPRD: correctly skip when we are skipping fewer values than in a vector by @@Mytherin in #12753 * Maintain prepared statement parameter types explicitly instead of converting into literals by @@Mytherin in #12759 * CLI .changes: use sqlite3_changes64 and sqlite3_totalchanges64 to prevent overflows by @@Mytherin in #12761 * Fix #12569: avoid truncating zeros that matter in format function by @@Mytherin in #12762 * Fix #12418: Remove .lint command in SQLite shell by @@Mytherin in #12763 * Fewer system calls in LocalFileSystem::ListFiles by @@Mytherin in #12769 * Support indexes in COPY DATABASE by @@Mytherin in #12768 * Issue #12600: Batched LEAD/LAG by @@hawkfish in #12770 * Issue #12600: Streaming Positive LEAD by @@hawkfish in #12685 * Add dl functionality for Windows by @@ccfelius in #12765 * Fix stale bot permissions, with @@szarnyasg by @@carlopi in #12782 * [CI] Stale bot: actually add permissions by @@carlopi in #12786 * Fix FILE_SIZE_BYTES test (again) by @@lnkuiper in #12779 * Fix extended tests in CI by @@carlopi in #12781 * Patch CentOS 7 EOL causing CI failure by @@samansmink in #12788 * Allow extensions to optionally add own description (on load) by @@carlopi in #12754 * Internal #2429: Shifted LEAD NULLs by @@hawkfish in #12791 * Issue #12784: Months Before Days by @@hawkfish in #12794 * bump vss by @@Maxxen in #12797 * Remove centos workaround by @@samansmink in #12801 * For compressed deletes in the undo buffer - count the actual size that will be written to the WAL when determining the auto-checkpoint threshold by @@Mytherin in #12803 * feat: new inet functions by @@panga in #12575 * [Dev] MakeExecutor::ResultCollectorIsBlocked less trigger-happy by @@Tishj in #12780 * Fix #12798: Add head file to fix debug build incomplete type error by @@zzachimed in #12810 * [Block Size] Switching to configurable block sizes by @@taniabogatsch in # 12691 * [Dev] Clean up the generate_serialization.py script a bit by @@Tishj in # 12823 * Remove micro extended from duckdb/duckdb by @@hmeriann in #12792 * [Python] Allow pathlib.Path to be provided to duckdb.connect by @@Tishj in # 12809 * remove .py scripts migrating to the sqlsmith by @@hmeriann in #12827 * Some expected error messages added by @@hmeriann in #12825 * [Fix] list_resize by @@taniabogatsch in #12832 * [Python] Python SQLLogicTester maintenance by @@Tishj in #12833 * [Python] Accept a list of Expression objects in DuckDBPyRelation.aggregate by @@Tishj in #12812 * [Copy Database] Don't include generated columns in the copied data by @@Tishj in #12835 * [Julia] Added Appender support for Int128, UInt128, and Base.UUID values by @@curtd in #12836 * [Python] Fix extraction of days/seconds/microseconds from timedelta object by @@Tishj in #12760 * [Python] Output a regular key -> value dict for hashable keys by @@Tishj in #12734 * [Dev] Change internals of StringUtil::GenerateRandomName by @@Tishj in # 12806 * [CSV] Adaptive Sniffer by @@pdet in #12698 * Fix LinuxRelease.yml after bump to Node 20 by @@carlopi in #12850 * Fix: Reduce repeated judgment in ties[i]. by @@Light-City in #12840 * add html_escape and html_unescape functions in inet extension by @@chrisiou in #11191 * Fix union struct implict cast by @@Maxxen in #12847 * Update httplib from v0.10.2 to v0.14.3 by @@lnkuiper in #12849 * Fix an issue where parameters would be promoted to NULL, incorrectly causing PREPARE to fail by @@Mytherin in #12844 * More expected error messages added by @@hmeriann in #12846 * Overload MIN/MAX/MAX_BY/MIN_BY to return the "top" N values by @@Maxxen in # 12834 * Fix #12789: list_zip support array by @@flashmouse in #12859 * [Python] Fix missing ConnectionException errors by @@Tishj in #12864 * Make equi_width_binning buckets even nicer by @@Mytherin in #12878 * Add a JSON pretty print function by @@PhictionalOne in #12398 * Add Metrics Support in the CAPI by @@maiadegraaf in #12498 * Update issue template by @@szarnyasg in #12880 * Fix regression in Parquet reader TryOpenNextFile by @@lnkuiper in #12848 * Improve performance of memory usage counters by @@yiyuanliu in #12751 * [Fix] No String Inline / Destroy Unpinned Blocks nightly run by @@taniabogatsch in #12884 * Fix LinuxRelease.yml CI by avoiding upload by @@carlopi in #12891 * Do not depend on manylinux extension in Python's CI by @@carlopi in #12895 * [Dev] Fix failing test in test_relation_api.cpp by @@Tishj in #12894 * [Dev] Skip test_run_pandas_with_tz on pandas <2.0.0 by @@Tishj in #12896 * Fix #12688: Julia SubString handling by @@dhanak in #12899 * [Python-Dev] Add DependencyException, throw earlier if PendingQuery fails by @@Tishj in #12888 * [Fix] Mixing block sizes and compression functions by @@taniabogatsch in # 12858 * Fix issue with list radix serialization by @@lnkuiper in #12887 * Respect limit during join order by @@Tmonster in #12851 * [CHORE]: Fix minor SQL test case by @@Tmonster in #12909 * Fixes for duckdb_constraints and information_schema constraint tables by @@Mytherin in #12914 * [Postgres Compatiblity] Support => to supply named parameters to functions by @@Tishj in #12913 * Pushdown table filters into probe based on min/max data found during hash build of hash joins by @@Mytherin in #12908 * Count nulls when detecting JSON structure by @@lnkuiper in #12883 * Fix #12870 - improve error message when encountering schema mismatches in COPY tbl FROM file.parquet by @@Mytherin in #12918 * Use different versions of snappy depending on the compiler by @@lnkuiper in #12889 * Fix Issues with type detection for Doubles/Floats/Decimals by @@pdet in # 12866 * Pass 'Unsecure_node_version' to allow node16 for a bit longer by @@carlopi in #12922 * Minor refactor for window_executor by @@ZhangHuiGui in #12924 * ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION:false for Android and Nightly by @@carlopi in #12930 * [Fix] Vector verification for dictionary expressions by @@taniabogatsch in # 12890 * Return smaller cardinality for Top N operator by @@jeewonhh in #12932 * Make GCC Happy Again by @@carlopi in #12937 * [Fix] More robust parquet metadata test by @@taniabogatsch in #12935 * Rework table filters, and for each row group only execute table filters if they can actually filter out any rows by @@Mytherin in #12938 * Issue #12941: Window Constant Results by @@hawkfish in #12943 * Expected errors 2053 by @@hmeriann in #12927 * Make ErrorData::Message() and ErrorData::RawMessage() const by @@Flogex in # 12885 * Feature #1272: Window Parallel Sink by @@hawkfish in #12907 * [Arrow] Add PhysicalArrowBatchCollector, a batched result collector for the ArrowQueryResult by @@Tishj in #12787 * [Python] Rework internals of object registration by @@Tishj in #12625 * [StreamQueryResult] Add ExecuteTask method to StreamQueryResult by @@Tishj in #12824 * Fix several CTE related issues by @@kryonix in #12948 * Label mark joins with convert to semi by @@Tmonster in #12916 * [Julia]: Added FixedDecimal support to Appender API by @@curtd in #12923 * Pushdown dynamically generated filters into MultiFileList, allowing partitions to be pruned by @@Mytherin in #12955 * Automatically materialize CTEs by @@lnkuiper in #12290 * Move checkpointing parallelism into TaskExecutor class, use that class for parallel union_by_name by @@Mytherin in #12957 * [Dev] Uncouple HTTPState from core by @@Tishj in #12912 * [Dev] Clean up TreeRenderer code by @@Tishj in #12716 * Swap build side and probe side base on cardinality AND width of build side. by @@Tmonster in #12755 * [Python] Use Set instead of List for the get_table_names stubs by @@Tishj in #12963 * Feature #1272: Window Validity Array by @@hawkfish in #12954 * [CSV Reader] Make glob reading more permissive to errors if ignore_errors is set by @@pdet in #12966 * [Python] Convert ENUM to np.array instead of pd.Categorical for fetchnumpy by @@Tishj in #12964 * Several hive partitioning fixes by @@Mytherin in #12970 * fix incorrect int32 appender by @@piever in #12956 * Add missing TransformCTE extraction to TransformRecursiveCTE by @@kryonix in #12968 * Expected errors 2053 by @@hmeriann in #12969 * Add Pyodide 0.26.1 (corresponding to Python 3.12) by @@carlopi in #12965 * Avoid adding a suffix to Parquet files when doing a partitioned write by @@Mytherin in #12972 * URL Encode/Decode Hive Partitioning Columns/Filters + add url_encode/ url_decode scalar functions by @@Mytherin in #12974 * Fix JSON extension Cmake by @@carlopi in #12975 * Clarify when it's necessary to refresh data and validity pointers by @@Giorgi in #12973 * Added new information_schema views - closes #4343 by @@prmoore77 in #12942 * Use jaro winkler similarity for finding similar entries in catalog by @@Mytherin in #12980 * Internal #2503: Streaming Window Reset by @@hawkfish in #12984 * Fix #12933: maintain insertion order in window when the window clause is empty (i.e. over ()) by @@Mytherin in #12982 * Feature #1272: Window Constant Sink by @@hawkfish in #12979 * Remove special case for '+' in URLs in httplib by @@lnkuiper in #12929 * Rebind prepared statements based on catalog versions by @@ywelsch in #12829 * Allow string stats larger than our default for parquet row group pruning by @@lnkuiper in #12928 * Add work-around for R client table function initialization back in under a config setting by @@Mytherin in #12990 * [Dev] Add getters/setters for the 'column_ids' of a LogicalGet by @@Tishj in #12971 * Accelerate Parquet en/decryption with HTTPFS extension by @@ccfelius in # 11720 * remove deprecated CDN invalidations by @@samansmink in #12997 * add large ingestions by @@hmeriann in #12949 * Fix shared_ptr issues in RowGroup and add locks to WAL initialization by @@Mytherin in #13000 * Add the ANY type and special null handling to the C API by @@taniabogatsch in #12785 * Improve error messages in the presence of subqueries by @@Mytherin in #13001 * Case-insensitive NULL casting in VARCHAR -> STRUCT casts by @@lnkuiper in # 13003 * TemporaryMemoryManager improvements by @@lnkuiper in #12931 * Make sure that empty and only white space headers have same treatment by @@pdet in #12994 * Push timestamp_tz to do direct casting if ICU is not loaded by @@pdet in # 12993 * [Dev] Add query to QueryRelation for logging by @@Tishj in #13004 * [Spark API] Fix group by compatibility issues by @@Tishj in #13005 * [CSV] Progress Bar for compressed files. by @@pdet in #12728 * Disable fixed size map in PartitionedTupleData by @@Mytherin in #13013 * Fix parallel creation and destruction of instances through the DBInstanceCache by @@Mytherin in #13010 * Avoid generating join filters for interval columns by @@Mytherin in #13014 * Extend least/greatest to support all types, and always return the same type as its input types by @@Mytherin in #13019 * Add sha1(varchar) -> varchar scalar function by @@bradlarsen in #13020 * Decimal to FloatingPoint: Avoid losing precision by splitting operation in two, more fixes and tests by @@carlopi in #12627 * Feature #1272: Segment Tree Finalize by @@hawkfish in #13002 * Create Delimiter Join and Delimiter Get via Relations by @@pdet in #12953 * Fix CTE/noalternativeverify issues by @@lnkuiper in #13024 * Fix bug in reworked fixed_size_map_t by @@lnkuiper in #13023 * Fix abs for floating point negative zero by @@Mytherin in #13025 * [Nightly-Bug] [CSV Reader] Use strings on header detection by @@pdet in # 13028 * Release GIL during DB instantiation by @@ywelsch in #13029 * GCC 4.8: add noexpr to 2 constructors by @@carlopi in #13030 * Fixes for LogicalType::ANY and fixed_size_map by @@Mytherin in #13035 * [Nightly] Fix for arrow appending fixed size lists by @@pdet in #13039 * fix storing persistent secrets in home dir by @@samansmink in #13033 * Internal #2534: IGNORE NULLS Threading by @@hawkfish in #13050 * [Julia] Only commit transaction on non-error (compat) by @@genericallyterrible in #13049 * Improve EXPLAIN output of Delim Joins and Delim Gets by @@kryonix in #12995 * Make client reuse threadsafe by @@samansmink in #13052 * Only remove the first occurrence of extension prefix by @@jeewonhh in #13057 * Fix incorrect overflow in left shift of unsigned number by @@Mytherin in # 13056 * Add tests for sha1 function by @@bradlarsen in #13064 * [Relation] ViewRelation could be created without an alias by @@Tishj in # 13054 * Support IN operator for LIST by @@Tishj in #12920 * [Python] Read from file-like objects with read_json by @@Tishj in #13040 * [Python] Fix lifetime issue with MaterializedRelation by @@Tishj in #12998 * PySpark sort by columns and DataFrame.getitem by @@khalidmammadov in #12981 * Fix trouble to compile with MSVC by @@annnei in #12579 * Create a dedicated RegisteredStateManager that manages client context states in a thread-safe manner by @@Mytherin in #13027 * DEBUG_STACKTRACE should not be enabled for release builds by @@Mytherin in # 13070 * Add blob overloads to crypto functions, and turn md5_number_lower and md5_number_upper into macros by @@Mytherin in #13068 * Internal #2534: IGNORE NULLS Threading by @@hawkfish in #13065 * Variable Integer Size Type by @@pdet in #13015 * Simplify binding of CALL statement by @@Mytherin in #13074 * Fix #13045: flatten in list_inner_product by @@Mytherin in #13076 * Fix for unpivot on zero columns by @@Mytherin in #13073 * Add support for overloading to macros by @@Mytherin in #13062 * Internal #2534: Windowed FILTER Threading by @@hawkfish in #13086 * Fixup #12579, remove pessimizing std::move by @@carlopi in #13089 * [CSV Reader] Fix when reading overbuffer values of csv files with extra delimiter by @@pdet in #13081 * Making error messages more clear for new line delimiter errors by @@pdet in #13082 * [CSV Sniffer] Sniffer can now sniff files with extra delimiters by @@pdet in #13083 * Remove test in Pyodide due to weird pandas interactions by @@carlopi in # 13091 * Add support for SQL-level variables by @@Mytherin in #13084 * Fix wasm CI and add missing template argument by @@carlopi in #13094 * Some clang tidy fixes around narrowing casts by @@carlopi in #13105 * chore: PushdownAggregate fast path by @@lichuang in #13098 * + fix list_distance & list_cosine_similarity execute fail when list i?? by @@flashmouse in #13090 * [Explain] Add EXPLAIN (FORMAT JSON) syntax by @@Tishj in #12967 * Expected errors 2053 by @@hmeriann in #13107 * [Bug] Enum types not being found if created in a schema by @@pdet in #13106 * Parquet reader can now read files with duplicate column names by @@hannes in #13111 * Prefer aliases over column-value functions in GROUP BY, and prefer error message when alias is used in an expression by @@Mytherin in #13118 * Disallowing DISTINCT, FILTER and ORDER BY for UNNEST by @@hannes in #13110 * Feature #1272: Windowed DISTINCT Sink by @@hawkfish in #13093 * adjust list value logic execute seq by @@flashmouse in #13114 * add CORE_EXTENSIONS build flag by @@samansmink in #13116 * [Python] Add missing options for read_csv by @@Tishj in #12872 * Fix merge conflict by @@Mytherin in #13126 * Retry on HTTP failure in extension install by @@Mytherin in #13122 * Clean up ChunkInfo when cleaning up a transaction by @@Mytherin in #13125 * typo: conjuction -> conjunction by @@qsliu2017 in #13127 * Add duckdb_result_error_type that returns the exception type of the error by @@Mytherin in #13131 * Keep track of user-provided configuration options, and compare them as well for database instance caching purposes by @@Mytherin in #13129 * Disable sniff_csv when enable_external_access is not set by @@hannes in # 13133 * Internal #2577: Window Atomic Finalize by @@hawkfish in #13117 * Internal #2577: Window Tree Allocation by @@hawkfish in #13119 * Fix optimizer error when dealing with IN with a single NULL parameter by @@Mytherin in #13139 * Internal #2597: Ragged Validity Array by @@hawkfish in #13137 * Create file with O_EXCL flag set. by @@mkaruza in #13123 * better CE for comparisons that use =, !=, <, <=, >, >=. by @@Tmonster in # 13130 * Use Slurm env vars to manage cpu and memory allocation if run inside Slurm HPC job by @@dirkpetersen in #12978 * Fix CTEFilterPusher optimization by @@kryonix in #13142 * Test with recent threadsanitizer by @@carlopi in #13124 * Fixes for RegexFindAll function errors and multibyte character support by @@chrisiou in #13108 * clang-tidy: rework Makefile and CI by @@carlopi in #13101 * Extension updates to also be logged via enable_http_logging by @@carlopi in #13087 * Hooks now get an error to indicate transaction/query success. by @@NiclasHaderer in #13136 * Use alias bind path in ORDER BY when running COLLATE by @@Mytherin in #13140 * Implementation of recursive JSONPath expressions by @@robert-s01 in #12991 * Adding option to disable materialized cte optimization by @@pdet in #13146 * [Python] Fix issue with native UDFs returning STRUCT items by @@Tishj in # 13147 * CGroups: fix compilation due to UB cast by @@carlopi in #13151 * test_expression.py: Skip throw related test on Windows by @@carlopi in # 13157 * Replaced while loop with if statement by @@NiclasHaderer in #13161 * ThreadSanitizer: Avoid spurious data race in InsertMatchesAndIncrementMisses by @@carlopi in #13179 * Add 'bugprone-narrowing-conversions' clang-tidy check by @@carlopi in #13180 * [C API] Return duckdb_value in duckdb_profiling_info_get_value by @@taniabogatsch in #13160 * Fix #13120: implement StructFilter::ToExpression by @@Mytherin in #13182 * Feature #1272: Windowed DISTINCT Sort by @@hawkfish in #13150 * Add native list_has_any implementation by @@Maxxen in #13163 * Add pragma extension versions by @@samansmink in #13063 * [C API] Add duckdb_scalar_function_set_volatile that allows changing FunctionStability of a scalar function by @@Mytherin in #13186 * [CSV Sniffer] Give preference to configurations that ignore the least amount of lines by @@pdet in #13188 * Fix #13017 - if grabbing the lock fails due to it not being supported, allow this in read-only mode by @@Mytherin in #13189 * Add test_collate_pivot: used to fail in 1.0.0, add to avoid regressions by @@carlopi in #13190 * Add catalog_error_max_schemas setting that toggles how many schemas we look at for "did you mean..." style error messages by @@Mytherin in #13191 * Fix build error when compiling with -DDISABLE_EXTENSION_LOAD=1 by @@whatsthecraic in #13194 * Add clang-tidy-diff script, and invoke it on PRs by @@carlopi in #13158 * [Explain (mostly internal only)] Change the way key-value pairs of information are stored for RenderTreeNodes by @@Tishj in #13109 * Fixup invocation of clang-tidy by @@carlopi in #13216 * [Julia] fix Vector and DataChunk all_valid() checks by @@aplavin in #13210 * Fix broken explain (format json) test by @@Mytherin in #13218 * Linux 32: avoid packaging Jemalloc, due to regression by @@carlopi in #13156 * feat(c): create value support by @@Mytherin in #13203 * feat(c): create value support by @@Mause in #12140 * fix: 13077: use tmp file for secret writes by @@devanbenz in #13170 * Filter paths before scripts/clang-tidy-diff.py by @@carlopi in #13220 * [CSV Reader/Progress Bar] Fixes by @@pdet in #13207 * Use poll in httplib by @@Mytherin in #13223 * Fix several fuzzer issues by @@Mytherin in #13240 * Fix #13238 - cannot return constant vector for volatile functions with more than one row as input by @@Mytherin in #13241 * Rename FORCE_CHECKPOINT to ALWAYS_CHECKPOINT by @@Mytherin in #13242 * C API test fix - correctly use length also in non-inlined case by @@Mytherin in #13243 * Rollback optimistic writers when all rows we have inserted are deleted by @@Mytherin in #13236 * Add serialization support and fix ToString of ChangeOwnershipInfo by @@Mytherin in #13233 * Fix #13200: Transactions that update tables need to keep the underlying row group collection alive to ensure we can safely clean-up by @@Mytherin in # 13230 * [CSV Reader] Comment Option by @@pdet in #13162 * Fix foreign key lookups from different search path by @@ywelsch in #13256 * Add aggregate function support to the C API by @@Mytherin in #13229 * Feature #1272: Window Task Blocking by @@hawkfish in #13249 * Implement #4318: add overload for pg_get_constraintdef by @@Mytherin in # 13258 * Several fuzzer fixes by @@Mytherin in #13261 * Correctly add profiling information from finalize events to operator timings in EXPLAIN ANALYZE by @@Mytherin in #13260 * Issue #13250: Zero Time Buckets by @@hawkfish in #13271 * Add delta to CI by @@samansmink in #13219 * Remove outadated test that fails somehow randomly and doesn't add much by @@carlopi in #13276 * Fix casts logic by @@carlopi in #13275 * Executor profiler fix by @@Mytherin in #13282 * Avoid clang-tidy-diff to check Python sources or extension folder by @@carlopi in #13279 * Enable yacc stack growing by @@Mytherin in #13280 * Fix #13272: correctly read signed integer stats for TIMETZ by @@Mytherin in #13283 * [Upsert] Fix crash caused by scanning an empty LocalTableStorage by @@Tishj in #13281 * Internal #2681: IEJoin Progress by @@hawkfish in #13284 * Move to latest duckdb-wasm (fixing COI compilation) by @@carlopi in #13257 * [DEV] CSV Tests Refactor and Sniffer decoupled of the vector size by @@pdet in #13231 * [TemporaryFileManager] Fix bug causing sizes of .block files to not be counted towards max_temp_directory_size by @@Tishj in #13278 * [CSV Reader] Properly cleanup invalid rows by @@pdet in #13262 * Disable website docs CI run by @@Mytherin in #13288 * Window Task Scheduling: avoid blocking tasks during GETDATA by @@Mytherin in #13290 * Fix #12582: correctly deal with empty grouping sets mixed with non-empty ones in lateral joins/correlated subqueries by @@Mytherin in #13291 * Reduce memory usage of test by @@Mytherin in #13292 * Move back from handrolled checkout to GH action + skip some verification in memory-intensive tests in CI by @@carlopi in #13296 * C API extensions by @@samansmink in #12682 * Secret settings cascade by @@samansmink in #13167 * Fix CMake for when folders are not git folders via git rev-parse --is-inside-work-tree by @@carlopi in #13315 * [Dev] Fix breakage caused by adjusting duckdb.h directly by @@Tishj in # 13321 * CAPI extensions: Fixup list of exported functions for wasm by @@carlopi in # 13320 * [Python Dev] Make cursor creation threadsafe, perform compaction on the internal vector by @@Tishj in #13319 * [Dev] Make (previously implicit) assertion explicit for DuckTransactionManager by @@Tishj in #13307 * [PySpark] Fix filter type checking and isin column expression return type by @@khalidmammadov in #13294 * Fix clang-format version in CONTRIBUTING.md by @@JelteF in #13324 * Expected errors 2053 by @@hmeriann in #13322 * Fix #13237: fix .mode insert float column output by @@flashmouse in #13308 * Add CUMULATIVE_CARDINALITY metric by @@maiadegraaf in #13234 * Fuzzer #3113: Lead Lag Shift by @@hawkfish in #13330 * Implemented list_extract with VectorOperations::Copy by @@Maxxen in #13313 * Issue template: Fix Arrow extension link by @@szarnyasg in #13333 * NO_PARTITION_COLUMNS option to skip partition writes for Parquet copy by @@ykskb in #12886 * Format Python README by @@szarnyasg in #13340 * Bug/Regression fixes by @@lnkuiper in #13317 * Enable unnesting lists of arrays by @@Maxxen in #13342 * Limit relation has wrong relation type by @@pdet in #13343 * [Dev] Remove redundant variable from SQLStatement by @@Tishj in #13341 * Internal #2722: Partition State Threading by @@hawkfish in #13350 * Adding hyperbolic trigonometric functions by @@nickgerrets in #13346 * CSV Sniffer - Error Messages by @@pdet in #13355 * Fix a typo by @@SYaoJun in #13360 * Add PySpark head, take and first functions by @@khalidmammadov in #13349 * [Explain] Add the GRAPHVIZ format for EXPLAIN statements by @@Tishj in # 13201 * Time cast: Have same behavior as Postgres by @@pdet in #13267 * Fix reordering semi joins by @@Tmonster in #13335 * Fix appian join tests by @@Tmonster in #13356 * remove sqlsmith patch by @@Tmonster in #13334 * Support Json Types in CSV Reader by @@pdet in #13359 * [CSV Sniffer] Date/Timestamp Sniffing adjustment by @@pdet in #13168 * WAL: Write pointers to optimistically written row groups directly, instead of copying over the data by @@Mytherin in #13372 * [Metrics] Add CUMULATIVE_ROWS_SCANNED and OPERATOR_ROWS_SCANNED by @@maiadegraaf in #13354 * [Transformer] Fix loss of named parameter data in recursive TransformStatement calls by @@Tishj in #13344 * Feature #1272: Window Distinct Merging by @@hawkfish in #13329 * Refine heuristic for flipping join sides by @@lnkuiper in #13399 * Update jemalloc and re-enable opt.retain by @@lnkuiper in #13370 * Add native list_has_all implementation by @@Maxxen in #13401 * Feature #1272: Window Distinct Indices by @@hawkfish in #13394 * [Arrow] Support consuming an "arrow_array_stream" PyCapsule by @@Tishj in # 13386 * Sample at least one value for the hyperloglog by @@Tmonster in #13383 * Fix issues with JSON map inference by @@lnkuiper in #13387 * Compressed materialization for joins by @@lnkuiper in #13402 * Python installation: Recommend using pip by @@szarnyasg in #13357 * [Python UDF] Filter NULL values before calling the user defined function by @@Tishj in #13358 * Add support for scalar function overloads to the C API by @@Mytherin in # 13409 * Fix unused variable, resolve warning turned error in the amalgamation CI by @@carlopi in #13408 * Add support for aggregate function overloads to the C API by @@Mytherin in # 13410 * Fix lossy double cast issue by @@lnkuiper in #13411 * Buffer manager: set handle readers after I/O so that any I/O exceptions don't leave the readers with an invalid value by @@Mytherin in #13417 * fix undefined symbol in httpfs for python by @@cyberjunk in #13420 * Fix dsdgen args by @@Tmonster in #13421 * [Arrow] Support producing an "arrow_array_stream" PyCapsule by @@Tishj in # 13418 * Disable jemalloc on 32bit through cmake, not through a CI script by @@lnkuiper in #13433 * allow changing the default persistent secret storage after initialization by @@samansmink in #13434 * Fixing hugeint cast to varint by @@pdet in #13268 * [Explain] Add the HTML format for EXPLAIN statements by @@Tishj in #13202 * fix for the issue 2698 by @@hmeriann in #13440 * Avoid rounding errors and simplify memory assignments in TemporaryMemoryManager by @@lnkuiper in #13439 * Pull up filters from and through explicit joins by @@lnkuiper in #13431 * [Metadata] Populate the expressions column of duckdb_indexes by @@Tishj in # 13415 * Use LossyNumericCast while reading memory limits by @@carlopi in #13450 * Label timestampTZ to timestamp cast as not revertible by @@Tmonster in # 13206 * Prevent the query thread from picking up query unrelated tasks if compiled with DUCKDB_NO_THREADS by @@NiclasHaderer in #13326 * [Dev] Skip test on lower pyarrow versions by @@Tishj in #13463 * Add TPCDS sf-100 benchmarks by @@hmeriann in #13205 * [Python] Let create_function cancel an open transaction instead of failing by @@Tishj in #13462 * [Dev] Mention non-unique indexes in UPSERT DO UPDATE SET error by @@Tishj in #13465 * Issue #13380: IN Invertible Casts by @@hawkfish in #13441 * Compilation Fixes for GCC 4.8 by @@hannes in #13474 * [Dev] Make Binder::Binder a private constructor by @@Tishj in #13475 * [Arrow] Accept objects that provide the __arrow_c_stream__ producer method by @@Tishj in #13425 * [Lambdas] Support N-ary lambdas in the list comprehension syntax by @@Tishj in #13389 * [METRICS] Add blocked_thread_time Metric by @@maiadegraaf in #13430 * Add ifdef to allow generating code with newer versions of Bison by @@Mytherin in #13477 * Clean up rendered text trees by @@Mytherin in #13476 * Make CMake target exports relocatable by @@cryos in #13312 * Disable delta extension on windows for now by @@Mytherin in #13485 * Regression runner - don't fail on HTTP error by @@Mytherin in #13486 * ExecutorTask: flush before finalizing task by @@Mytherin in #13482 * [Metrics] Rework Optimizer Metrics by @@maiadegraaf in #13480 * Support collations in ordered aggregates by @@Mytherin in #13491 * ci: minor optional cleanup for cibuildwheel config by @@henryiii in #13496 * chore: remove wheel dependency by @@henryiii in #13495 * Prefer depth-first plan evaluation for unions and joins by @@lnkuiper in # 13447 * Fix #13472: get correct WAL location for windows long paths by @@Mytherin in #13487 * Make error message more explicit for when trying to parse SQL types. by @@pdet in #13376 * [Arrow] Add UUID and JSON extension types by @@pdet in #13446 * Apply extension patches via 'patch' instead of 'git apply' by @@carlopi in # 13488 * Feature #1272: Window Distinct Tree by @@hawkfish in #13503 * Scalar macro default params by @@Alex-Monahan in #13494 * Add Varint to AllTypes() by @@pdet in #13388 * [ADBC] Support creation and ingestion into temporary tables. by @@pdet in # 13426 * Add support for registering custom casts (and types) through c api by @@Maxxen in #13499 * Bump sqlite & mysql by @@carlopi in #13515 * Switch floating point division and modulo to use IEEE semantics for division by zero by default, and add ieee_floating_point_ops setting that can be used to revert back to old behavior by @@Mytherin in #13493 * [C API] Get all metrics by @@taniabogatsch in #13497 * Tuning ART indexes for duplicate values by @@taniabogatsch in #13373 * Feature #1272: Windowed Quantile Tree by @@hawkfish in #13516 * [Auto Loading] Autoload extension settings by @@Tishj in #12790 * JSON bugfixes and new functions by @@lnkuiper in #13481 * Make changes to relation.hpp backward compatible with older API via default parameters by @@carlopi in #13522 * Preserve cardinality information by @@jeewonhh in #13517 * Return an error when multiple rows are returned from a scalar subquery by @@Mytherin in #13514 * When referencing optimistically-written blocks in the WAL, we need to fsync the main database file before writing the WAL to ensure all changes have made it to disk by @@Mytherin in #13526 * Remove assertion in compressed file system by @@Maxxen in #13531 * Skip empty files in single-threaded CSV reader by @@pdet in #13528 * Disallow ordering by non-integer literal by default - and add the setting order_by_non_integer_literal to revert to the previous behavior by @@Mytherin in #13532 * Produce an empty list result for enum_range(NULL::enum_type) by @@pdet in # 13530 * [CSV Reader] Fix for invalid unicode in header by @@pdet in #13518 * Issue #13525: Window Local States by @@hawkfish in #13538 * Linux32 also to regular ci by @@carlopi in #13547 * Feature #1272: Windowed Distinct Tree by @@hawkfish in #13527 * Bump GitHub workflows to their latest versions by @@deining in #13393 * Add map_contains, map_contains_entry and map_contains_value functions, list_position now returns null. by @@Maxxen in #13406 * Numeric casts: forbid NumericCasts for float/double, add Lossy and Exact equivalent to make syntax explicit by @@carlopi in #13546 * Fix clang-tidy on insertion_order_preserving_map.hpp: Alter order of statements by @@carlopi in #13556 * Add http proxy by @@samansmink in #13368 * move the inet extension out-of-tree by @@chrisiou in #13085 * Fix compilation: install extension needs an instance now by @@Mytherin in # 13565 * Initialize (potentially) empty buffer by @@lnkuiper in #13564 * HTTP glob test fix by @@Mytherin in #13563 * Remove patches (vss and substrait), bump other repositories by @@carlopi in #13551 * Disable dynamic filter pushdown for right semi joins by @@Mytherin in #13562 * postgres_scanner: bump & remove patches by @@carlopi in #13567 * Optimistic write to WAL: cannot write block pointers if we have in-memory updates to transaction local storage by @@Mytherin in #13577 * Add all CAPI functions to extension api for now by @@samansmink in #13568 * Minor fixes for DuckDB-Wasm by @@carlopi in #13566 * Fix for vector size 2 tests by @@Mytherin in #13569 * [CSV Sniffer] Date Adjustment by @@pdet in #13573 * Fix issues found by alternative verify by @@Mytherin in #13578 * Set version info for duckdb.dll by @@Giorgi in #13557 * Update storage info by @@szarnyasg in #13483 * Constrain size of estimated cardinality returned when operator is not initialized by @@Mytherin in #13574 * Unify task (un)blocking in physical operators by @@lnkuiper in #13559 * [CSV Reader] Fix lock issue on Global CSV Reader by @@pdet in #13560 * TIMETZ group by: collations now no longer always return VARCHAR by @@Mytherin in #13571 * Internal #2850: Window Local States by @@hawkfish in #13581 * Two steps upload action by @@carlopi in #13582 * Use new HLL implementation in DistinctStatistics by @@lnkuiper in #13489 * Remove inet from extension update test by @@Mytherin in #13589 * [METRICS] Small fixes by @@maiadegraaf in #13575 * Fix #13537: correctly maintain parameter count when rebinding a prepared statement by @@Mytherin in #13583 * Various nightly test fixes by @@Mytherin in #13584 * IE Join: turn these into atomics to prevent tsan from tripping up in GetProgress by @@Mytherin in #13586 * Fix HTTPFS tests by @@Mytherin in #13588 * [PyArrow] Fix issue with passing timestamp types to filters by @@pdet in # 13593 * Fuzzer fixes by @@Tmonster in #13596 * Automatically call malloc_trim to reduce unused outstanding allocations by @@lnkuiper in #13590 * Fix for joins and FSST on 32-bit configurations by @@Mytherin in #13594 * Casting to Bit is no longer Invertible by @@Tmonster in #13595 * Switch in-tree DuckDB extension to use DuckDB's semver tag by @@samansmink in #13591 * List has bind fix by @@Maxxen in #13600 * Do not include version.rc file on MINGW by @@Giorgi in #13601 * [Python RelAPI] Throw an error if trying to use a invalid argument in read_csv by @@pdet in #13597 * More nightly test fixes by @@Mytherin in #13605 * Optimistic write to WAL: we cannot optimistically write block pointers if there are indexes by @@Mytherin in #13610 * Block verification run and FixedSizeBuffer fix by @@taniabogatsch in #13607 * [Test] Tighter ART storage regression boundaries by @@taniabogatsch in # 13612 * Disable JEMALLOC_HAVE_MADVISE_HUGE in jemalloc by @@lnkuiper in #13608 * Avoid adding buffers that will be destroyed to the eviction queue by @@lnkuiper in #13606 * Track overflow strings in PartialBlockManager during optimistic write by @@Mytherin in #13618 * [Fix] Avoid index deletion after catalog exception by @@taniabogatsch in # 13627 @ text @d1 1 a1 1 # $NetBSD: Makefile,v 1.4 2024/08/25 06:18:28 wiz Exp $ d3 1 a3 1 DISTNAME= duckdb-1.1.0 @ 1.4 log @*: replace CMAKE_ARGS with CMAKE_CONFIGURE_ARGS @ text @d1 1 a1 1 # $NetBSD: Makefile,v 1.3 2024/06/05 18:22:39 ryoon Exp $ d3 1 a3 1 DISTNAME= duckdb-1.0.0 @ 1.3 log @databases/duckdb: Update to 1.0.0 Changelog: DuckDB 1.0.0 "Nivis" What's Changed * Fix move constants optimization by @@gitccl in #12189 * FALLOC_FL_PUNCH_HOLE requires GLIBC 2.18 or above - check for this using an #ifdef by @@Mytherin in #12193 * Fix cmake install for shared_ptr headers by @@carlopi in #12194 * Fix minor warnings by @@carlopi in #12195 * Install .ipp files in addition to the usual .hpp files by @@smonkewitz in # 12198 * Set a default value to the connection param in stubs by @@tm-drtina in # 12207 * Fix #12190: add SYSTEM to set of reserved database names by @@Mytherin in # 12206 * Add enable_view_dependencies which defaults to false by @@Tishj in #12209 * [Python] Fix replacement scans incorrectly finding duckdb connection method objects by @@Tishj in #12208 * [CI] Diff against the right remote + branch in Regressions.yml - Regression Test new micro benchmark by @@Tishj in #12106 * [Python] Fix bug where enable_external_access was not being respected by the replacement scan by @@Tishj in #12224 * Remove outdated CI for extensions, check duckdb/extension-ci-tools by @@carlopi in #12229 * Python: Avoid packaging for both 3.7 on OSX and MacOS 11 by @@carlopi in # 12236 * Issue #12215: AsOf Predicate Pushdown by @@hawkfish in #12238 * [DependencyManager] Don't block ADD COLUMN statements if there are dependencies. by @@Tishj in #12226 * [Python] Add check for 'params' to table_function by @@Tishj in #12233 * Extension installing/updateing fixes by @@samansmink in #12221 * Move excel extension out of tree by @@carlopi in #12123 * Fix #12225: revert OVERWRITE_OR_IGNORE to previous behavior, move new behavior to OVERWRITE flag by @@Mytherin in #12240 * Fix warning on unannotated fallthrough by @@carlopi in #12244 * Fixup staged uploads /3 by @@carlopi in #12245 * [Python] fix build when BUILD_HTTPFS set by @@paparodeo in #12223 * Use --always option of git describe for extension tags. by @@carlopi in # 12253 * [Docs] Fix up examples/python/duckdb-python.py by @@Tishj in #12255 * [Fix] Skip lazy WAL creation test for alternative verification by @@taniabogatsch in #12258 * add missing virtual destructor by @@samansmink in #12266 * Add storage callbacks for checkpoint start and end by @@bleskes in #12260 * Do not prefix error messages with an unknown type by @@bleskes in #12265 * Fix minor duckdb_extensions table function bug by @@samansmink in #12269 * C API: Adding deprecation and move notices to duckdb.h and a test case by @@hannes in #12259 * Issue #12252: APPROX_QUANTILE Array Argument by @@hawkfish in #12271 * Turn InternalException into NotImplementedException in COPY FROM DATABASE by @@Mytherin in #12264 * Add descriptions for vss and delta extensions by @@carlopi in #12267 * [C-API] Properly handle exceptions caused by name collisions in duckdb_register_table_function by @@Tishj in #12257 * Fix for multifilereader extra_columns feature by @@samansmink in #12281 * Add enable_macro_dependencies which defaults to false by @@Mytherin in # 12291 v0.10.3 Bugfix Release Highlights Even though this is "only" a bug fix release, there have been some major areas of work that warrant a separate mention: * We have added a feature to update extensions using the UPDATE EXTENSIONS syntax #11677 * There have been some serious internal improvements around checkpointing, most notably, checkpoints can run while other connections are reading, and no longer block new connections while checkpointing #11918. Also, FORCE CHECKPOINT no longer actively cancels transactions, it now waits until it can checkpoint #12061 * DuckDB now has native support to load data from HuggingFace using the hf:// prefix #11831 * We have slightly changed NULL casting behaviour with the MAP type #11745 * The Java JDBC driver has been moved to its own repo: https://github.com/ duckdb/duckdb-java #11873 * DuckDB now cleanly compiles with -Wconversion and all conversions are actually being checked #11716, #11673 What's Changed * Add setting to control the maximum swap space by @@Tishj in #10978 * [Python][Dev] Dynamically generate the Connection wrapper methods by @@Tishj in #11202 * Fixes duckdb wasm by @@carlopi in #11688 * Checked conversions between signed and unsigned integers by @@hannes in # 11673 * Bump Julia to v0.10.2 by @@Mytherin in #11700 * Minor improvements to sql_reduce script by @@Mytherin in #11701 * Properly avoid build-time dependency on Python by @@carlopi in #11713 * Test dockerized compilation in Alpine:latest and Ubuntu:20.04 by @@carlopi in #11708 * [COPY CSV] Enable TIMESTAMP_TZ formats by @@Tishj in #11711 * Full conversion warnings / checks by @@hannes in #11716 * [Safety] Add safety checks to shared_ptr access by @@Tishj in #11696 * Remove bound_defaults from BoundCreateTableInfo by @@Mytherin in #11721 * Improve mkdir error reporting by @@Mytherin in #11723 * [Dev] Fix failing CI in Python SQLLogicTest Runner by @@Tishj in #11724 * More docker tests, fix compilation up to C++23 standard by @@carlopi in # 11725 * Upload staging: from 'git describe --tags' to 'git log -1' by @@carlopi in # 11715 * Internal #1848: Window Progress by @@hawkfish in #11702 * Remove BoundConstraint from the TableCatalogEntry by @@Mytherin in #11735 * Implicit Cast for any Date/Timestamp by @@pdet in #11733 * feat: rewrite which_secret() into a table function by @@stephaniewang526 in #11726 * [Map] Rework MAP creation method behavior when input is NULL by @@Tishj in # 11730 * [Dev] Always use SQLStatement->Copy() when ALTERNATIVE_VERIFY is defined by @@Tishj in #11732 * Reconstruct Error Messages for Flush Cast by @@pdet in #11736 * Getting Rid of Value.TryCast in the CSV Sniffer by @@pdet in #11717 * Fix Join order optimizer so that plan generation is always via the most current entry in the DP table. by @@Tmonster in #11719 * fix(py): support DuckDBPyType#children for array and enum by @@Mause in # 11754 * Consider not null values when doing export database by @@pdet in #11679 * Add missing space in error message by @@szarnyasg in #11759 * Allow to build python packages without c++ sources by @@carlopi in #11758 * No Mark to Semi join conversion in statistics propagation by @@Tmonster in # 11596 * Hive partitioned write: lazy partitioning initialization by @@Mytherin in # 11765 * Hive partitioning: avoid calling CreateDirectories for every flush, instead create the directory for a partition only when that partition is instantiated by @@Mytherin in #11777 * [Parquet] Support reading the non-standard NULL ConvertedType by @@Tishj in #11774 * Only store CSV Errors if we are doing rejects table, otherwise just ignore it. by @@pdet in #11763 * CI: Add job for 'expected behavior' label by @@szarnyasg in #11784 * Move recursive_query_csv.test to slow test by @@pdet in #11770 * [StatementVerifier] Fix up issues in ToString implementations of classes derived from SQLStatement by @@Tishj in #11625 * Hive partitioning: make OVERWRITE_OR_IGNORE remove files on local file systems by @@Mytherin in #11787 * [ODBC] Add ODBC Test for Database Reconnection and Data Persistence by @@maiadegraaf in #11783 * Correctly parse dollar-quoted strings in sqlite3_complete and linenoise by @@Mytherin in #11789 * Add a configurable compression_level parameter to the parquet writer by @@Mytherin in #11791 * Close file after file lock failure by @@awitten1 in #11795 * Python: Add missing options to write_parquet by @@jzavala-gonzalez in #11790 * [PythonDev] Fix up failing tests in CI by @@Tishj in #11801 * Fix static bitpacking_width_t FindMinimumBitWidth(T *values, idx_t count) in class BitpackingPrimitives by @@Lloyd-Pottiger in #11757 * Add note on CMAKE_BUILD_PARALLEL_LEVEL by @@mlafeldt in #11808 * Elaborate on internal errors by @@szarnyasg in #11816 * Fix #11756: Don't throw exception on CREATE UNIQUE INDEX IF NOT EXISTS if index already exists by @@ewencp in #11821 * Python CI fixes: skip two tests by @@carlopi in #11818 * Fix #11798 - lateral join parameters should not be visible in views by @@Mytherin in #11825 * Fix #11804: make sure json_type can check null by @@lnkuiper in #11807 * Fixing performance regression in [u]hugeint cast by @@hannes in #11829 * [Dev] ClientContextWrapper yak shaving by @@Tishj in #11830 * [Python] Add checkpoint method, improve shutdown experience by @@Tishj in # 11810 * [Benchmark] Enable benchmarking result collection by @@Tishj in #11529 * [DependencyManager] Create dependencies between foreign key tables and primary key tables. by @@Tishj in #11524 * [Python] Synchronize defaults of DuckDBPyRelation method fetch_df_chunk by @@Tishj in #11834 * Internal #1888 TIMETZ Collation Keys by @@hawkfish in #11861 * Removing old code that used to check if a buffer was the last buffer from the file handler by @@pdet in #11846 * Use ToSQLString() in ConstantFilter for escaped filter output by @@rcurtin in #11797 * [StatementVerifier] Add ToString for every remaining SQLStatement, is pure virtual now by @@Tishj in #11788 * Pushdown Tables Types to CSV Scanner by @@pdet in #11792 * [Python Dev] Fix shift between requirements-dev.txt and pyproject.toml before-test section by @@Tishj in #11863 * Join order optimizer asan bug Follow up by @@Tmonster in #11794 * BugFix: Introducing Introducing Delim Joins and Delim_Get(s) should respect positionally by @@Tmonster in #11812 * Provide the native OID of PG type in pg_type by @@goldmedal in #11746 * Move JDBC (Java) Driver to Separate Repo by @@hannes in #11873 * Link Java client in issue template by @@szarnyasg in #11877 * Change specificity of sniffed types to check time related types earlier by @@pdet in #11878 * fix complex top n test case for constant vector verification by @@Tmonster in #11882 * [Dev] Merge overloads for HUGEINT cast functions by @@Tishj in #11879 * Make " default for quote and " default for escape by @@pdet in #11880 * Set secret directory to a test directory when running sqllogictest by @@Mytherin in #11885 * Bugfixes by @@lnkuiper in #11785 * [Map] Rework interaction (entries, keys, values, extract) of NULL MAPs by @@Tishj in #11745 * Add case when expression for grouping sets when collations are used. by @@Tmonster in #11884 * Internal #11892: Interval Quarter Keyword by @@hawkfish in #11898 * HTTP Logging by @@lnkuiper in #11771 * [Dev] Use strings in the SQLLogicTest REQUIRE calls so they are visible with -s by @@Tishj in #11714 * [Dev] Fix a SerializationException on CopyInfo by @@Tishj in #11902 * MultiFileReader refactor by @@samansmink in #11806 * Allow checkpoints to run while other connections are reading, and no longer block new connections while checkpointing by @@Mytherin in #11918 * Allow converting TIMETZ to Arrow by @@LoganDark in #11906 * Issue #11894: MIN/MAX_BY DECIMAL Casting by @@hawkfish in #11912 * Issue #1917: WinNode 22 Compilation by @@hawkfish in #11913 * [Relation] Add MaterializedRelation by @@Tishj in #11835 * Enable purging of BufferPool pages based on time-since-last-unpinned by @@jkub in #11441 * Correctly render duckbox for empty results by @@Mytherin in #11920 * Always store transactions that had errors during the commit phase by @@Mytherin in #11929 * More anonymous struct zapping in RE2 by @@hannes in #11956 * Add the corrupt block location to the exception by @@Vegetable26 in #11966 * Fix assertion in bitpacking by @@nickgerrets in #11955 * [Python] Add CoalesceOperator to Python Expression API. by @@Tishj in #11941 * CMake: Handle git failures on invalid inputs better by @@carlopi in #11951 * Internal #2005: DISTINCT ORDER BY by @@hawkfish in #11967 * Fix overlooked function argument rename that leads to seg faults. by @@smonkewitz in #11969 * [Nightly] Block size test fixes by @@taniabogatsch in #11972 * Optimizing InsertionSort by reducing the size of the comparison by @@gitccl in #11964 * [Python] Keep referenced Python objects alive by @@Tishj in #11761 * Move mysql_scanner into main duckdb CI by @@carlopi in #11999 * Fix CURRENT_SETTING with a NULL string arg by @@gitccl in #12015 * Issue #12009: APPROX_QUANTILE NULL List by @@hawkfish in #12014 * Issue #12003: TIMESTAMP Stack Overflow by @@hawkfish in #12012 * fix extension load error message grammar by @@softprops in #11994 * [Python] Fix InternalException from scanning Polars DF with no columns by @@Tishj in #11982 * Issue #11959: TIMESTAMPTZ >= DATE by @@hawkfish in #11987 * More fixes for RE2 to pass CRAN tests by @@hannes in #11978 * chore: update exception message by @@stephaniewang526 in #11965 * Issue #12005: RESERVOIR_QUANTILE DECIMAL Binding by @@hawkfish in #12013 * [Python] Grab the GIL in the destructor of PyFilesystem by @@Tishj in #11980 * [Python] Make the NumPy module optional, not throwing if it's not installed by @@Tishj in #11981 * Add support for HuggingFace to httpfs by @@samansmink in #11831 * [Fix] lambda binding in ALTER TABLE statements by @@taniabogatsch in #11976 * Distinguish between exact and case insensitive matching JSON keys in json_structure by @@lnkuiper in #11948 * Rework index binding by @@Maxxen in #11867 * Issue #11995: TIMESTAMP Rounding by @@hawkfish in #12011 * Fix sample serialization by @@Tmonster in #12025 * Correctly skipping errors when ignore_errors is set and we have columns with escaped values by @@pdet in #12027 * Update comment to reflect correct data state post-compression by @@wangxuqi in #12022 * Fix ordering issue with nested list type by @@gitccl in #11937 * Adding Fix to properly pass timestamp/date formats in the relational API for CSV Files by @@pdet in #12029 * Add more MultiFilereader features/hooks by @@samansmink in #11984 * Rethrow serialization errors by @@carlopi in #12030 * Move yyjson into core by @@Maxxen in #11998 * Bugfixes + large allocation hardening by @@Maxxen in #12028 * Ensure HT capacity is greater than lower bound by @@lnkuiper in #12039 * Fix materialized CTE plan issue by @@kryonix in #11874 * Fix some fuzzer issues by @@hannes in #12043 * [Fix] Return NULL for deprecated getter calls in the C API by @@taniabogatsch in #12035 * Grab checkpoint lock during storage metadata reads by @@Mytherin in #12053 * Issue #12041: TIMETZ Parquet Nanoseconds by @@hawkfish in #12052 * Parquet: Correctly return min/max string stats if empty by @@lnkuiper in # 12054 * Even more fuzzer fixes by @@Maxxen in #12050 * [Fix] Silent constraint violation error when destroying the appender in the C API by @@taniabogatsch in #12051 * Add "Tags" support to catalog entries by @@Maxxen in #12044 * Rework FORCE CHECKPOINT - instead of actively cancelling transactions it now blocks until it can checkpoint by @@Mytherin in #12061 * Aggregation bugfixes by @@lnkuiper in #12055 * [Fix] Disable test for block size nightly run by @@taniabogatsch in #12062 * Bind art index in local storage by @@Maxxen in #12064 * Cast keys to VARCHAR before creating JSON from MAP by @@lnkuiper in #12065 * [Python] Add pyspark hash and organize unit tests by @@mariotaddeucci in # 11935 * Check context.interrupted during force checkpoint by @@Mytherin in #12068 * [Fix] Lazy WAL creation by @@taniabogatsch in #12049 * Test docker images: improvement and connected fixes by @@carlopi in #12026 * More fuzzer fixes by @@hannes in #12045 * [Python] Add pyspark null functions by @@mariotaddeucci in #11940 * CI fixes: unused variable & toolchain version by @@carlopi in #12083 * Add autoloading for delta extension by @@samansmink in #12063 * S3FileHandle Destructor should call Close() conditionally by @@onderkalaci in #12031 * [Fix] Internal segment tree exception in on conflict clause by @@taniabogatsch in #12084 * Remove ClientContext usage in Checkpoint Reader by @@Mytherin in #12076 * Fixed Parquet crash on missing dictionary by @@hannes in #12085 * [Fix] Add lambda binding to the HAVING binder by @@taniabogatsch in #12070 * Decimal/Time implicit casting + Multi-Error store in Flush by @@pdet in # 11848 * [Testing Infra Fix] Make input data chunks immutable in the vector verification tests by @@taniabogatsch in #12088 * Correctly rewrite correlated columns inside window functions by @@Mytherin in #12087 * Fix #11780 - handle qualifications in ORDER BY of ARRAY clause by @@Mytherin in #12090 * Nightly CI fixes by @@Mytherin in #12093 * Change ExtensionOptimizer input by @@Maxxen in #12094 * Fix for issue related to the execution of union by all from .sql in Python by @@pdet in #12098 * yyjson bump version to 2020 by @@carlopi in #12072 * [Dev] Collect CatalogEntry Dependencies during Binding by @@Tishj in #11493 * Internal #2040: ICU Collation Serialisation by @@hawkfish in #12077 * Run python tests in Pyodide build by @@cpcloud in #11914 * Add support for type modifiers on extension types by @@Maxxen in #12081 * Bump extensions by @@carlopi in #12107 * fix huggingface credential_chain autoload issue by @@samansmink in #12112 * Fix fuzzer issue 2690 by @@lnkuiper in #12108 * Throw exception in case of WAL failure instead of only printing a message by @@Mytherin in #12091 * Change type of columns from sniff_csv to list of structs by @@pdet in #12099 * [Python][Dev] Skip statements with decorators (only if, skip if) in the Python SQLLogicTester by @@Tishj in #12102 * Mark unspecialized C++ Append template as delete by @@j1ah0ng in #12116 * SQLLogicTest - skip these tests now that we have dependencies between views by @@Mytherin in #12118 * Correctly determine if we need to scan flat vectors in all cases - and add an enum to clarify code by @@Mytherin in #12119 * Avoid signed integer overflow in sequence generation by @@Mytherin in #12120 * Use Binder::BindCreateTableCheckpoint in WAL ReplayCreateTable by @@Mytherin in #12121 * Avoid checking if wal is set directly and call GetWALSize instead - a WAL might be present even if wal is not set by @@Mytherin in #12124 * Call StringVector::AddString here for when inlining is disabled by @@Mytherin in #12125 * Minor fixes for vsize=2 tests by @@Mytherin in #12126 * Internal #2078: Nested Nulls First by @@hawkfish in #12131 * Bump extensions, part 2 by @@carlopi in #12122 * Internal #2081: Window Distinct Reset by @@hawkfish in #12130 * Read scan count once instead of once per vector to avoid issue where scan counts between vectors could become mis-aligned in concurrent scenarios by @@Mytherin in #12135 * Extension Updating by @@samansmink in #11677 * Move pyodide from repository_dispatch to NightlyTests.yml by @@carlopi in # 12153 * [Storage] Add storage_compatibility_version to control for what version the DB has to be serialized. by @@Tishj in #12110 * Allow quotes to be escaped in JSON path by @@lnkuiper in #12033 * [Python] Fix issue in the SQLLogicTestRunner implementation by @@Tishj in # 12155 * Higher memory limit for test by @@lnkuiper in #12158 * Fix internal error of list_zip and map_concat by @@gitccl in #12086 * fix row format of arrays larger than vector size with null by @@Maxxen in # 12143 * Issue #12136: Streaming Window Structs by @@hawkfish in #12150 * Set max vector size to 128GB instead of 4GB by @@Mytherin in #12144 * Pass prepared statement parameters to OnExecutePrepared callback by @@Mytherin in #12156 * In string to list try_cast - set the target index to NULL, not the source index by @@Mytherin in #12160 * More Nightly CI Fixes by @@Mytherin in #12154 * Fixing unchecked malloc() calls in Parser and elsewhere by @@hannes in # 12162 * Modify the pandas analyzer code to always respect the sample size by @@pdet in #12097 * Allow community extensions: add setting and keys by @@carlopi in #12152 * Fixing parquet dictionary / data page offset bug by @@hannes in #12109 * small fix to extension origin checks and direct installing over http by @@samansmink in #12165 * [DependencyManager] Provide details in case of a DROP statement that needs CASCADE. by @@Tishj in #12159 * Remove UnsafeNumericCast in create_sort_key by @@Mytherin in #12168 * [Dev] enable_verification now serializes for compatibility version 'latest' by @@Tishj in #12157 * [Relation] Disable creating a VIEW from a MaterializedRelation by @@Tishj in #12163 * Move community keys to proper values by @@carlopi in #12175 * Remove release assertions timeout by @@Mytherin in #12176 * Internal #2095: Streaming Window Structs by @@hawkfish in #12173 * [CSV Reader] Bug-fix related to skip parameter over vector size in the sniffer by @@pdet in #12167 * Expression rewrite filter pushdown for dates by @@Tmonster in #12056 * [Python] Throw if replacement scan is attempted on cross-connection DuckDBPyRelation by @@Tishj in #12169 * [Fix] Correctly allocate the ARRAY target child vector in a MAP function by @@taniabogatsch in #12111 * Remove java from CI invoker by @@hannes in #12182 * Mark correct database as modified in CreateIndex by @@Mytherin in #12183 v0.10.2 Bugfix Release SQL Modifications This release has a number of bug fixes that change SQL semantics in a few edge cases: * Nested Boolean Comparisons now have consistent NULL comparison semantics - #11496 * Structs with non-matching keys require explicit casts when compared or combined - #11396 What's Changed * Bump julia version & fix release script for sub-versions > 9 by @@Mytherin in #11225 * Flatten Rewrite by @@maiadegraaf in #11223 * ORDER BY ColumnNumber with Collations by @@tiagokepe in #11139 * Fix differences to implementation for to_parquet, write_parquet, to_csv, write_csv, Expression.alias, DuckDBPyRelation.map by @@binste in #11135 * Issue template: Ask for MWEs by @@szarnyasg in #11192 * Cleaning up FSST: Remove unused AVX512 code by @@hannes in #11222 * Fix #11211 - correctly fill in string_t padding for bit type by @@Mytherin in #11231 * Fix #3391: Stop creating background threads if the thread constructor throws an exception by @@Mytherin in #11236 * R_CMD_CHECK: Pin to duckdb/duckdb-r 0ed106a71c by @@carlopi in #11245 * Add support for HEX(BLOB) by @@Mytherin in #11243 * Remove no_vector_verification in Map Subscript Test by @@maiadegraaf in # 11242 * Update logos in README by @@szarnyasg in #11256 * Ignore user defined parameters that change names or types of csv columns in sniffer's prompt. by @@pdet in #11257 * [Python] Fix error caused by looking up a TypeCatalogEntry without an active transaction. by @@Tishj in #11255 * [Fix] Fuzzer issue in list_select by @@taniabogatsch in #11248 * [Parquet] Support for LZ4 Compression by @@hannes in #11220 * Fix #11254: Add missing includes to terminal by @@Mytherin in #11265 * Issue #10867: AsOf Predicate Pushdown by @@hawkfish in #11233 * Fix plan cost runner regression script by @@Tmonster in #11129 * Check if we need to throw any remaining errors at end of CSV scanning by @@pdet in #11276 * Allow duplicate names in json objects when ignore_errors is true by @@lnkuiper in #11271 * Do not surround JSON with quotes in sqlite shell output by @@lnkuiper in # 11268 * add TRIM support to virtual filesystem, and implementation on linux by @@jkub in #11258 * Perform direct write operation if input data are larger than buffer size by @@quentingodeau in #11203 * Fuzzer fixes by @@lnkuiper in #11286 * Compile spatial also for rtools by @@carlopi in #11291 * allow injecting custom BufferManager implementation by @@jkub in #11215 * Default to RECORDS in JSON reader if more than one column is specified by @@lnkuiper in #11295 * Add support for materialized CTEs in INSERT/UPDATE/DELETE statements by @@kryonix in #10878 * Only throw exception if je_mallctl fails in DEBUG mode by @@lnkuiper in # 11303 * Fixing casting issue in generators by @@hannes in #11304 * Rework FileSystem::OpenFile call, and add FILE_FLAGS_NULL_IF_NOT_EXISTS by @@Mytherin in #11297 * Fix potential UB when list() aggregate is used in combination with other arena using aggregate functions by @@Maxxen in #11306 * Fix #11293 - for ARRAY([subquery]) explicitly push the ORDER BY of the underlying subquery into the array aggregate by @@Mytherin in #11316 * Fix #11281: explicitly select column types of information_schema tables for all columns, even if they are always NULL by @@Mytherin in #11317 * Fixup py upload by @@carlopi in #11308 * Issue #11279: TIMESTAMP => TIMESTAMPTZ by @@hawkfish in #11320 * Fix null pointer exception when rolling back updates if the rollback was caused by an OOM by @@Mytherin in #11309 * Fix #11283 - report consistent foreign key constraint name in information_schema by @@Mytherin in #11318 * Fix #11294 - avoid applying Filter Pushdown optimization for UNION/EXCEPT without ALL by @@Mytherin in #11315 * Fix #10695 - handle ? prepared statement parameters correctly for POSITION (x IN y) by @@Mytherin in #11314 * Windows CLI - emit UTF8 directly using SetConsoleOutputCP(CP_UTF8) if possible by @@Mytherin in #11324 * Fix #11319: use modulo when computing day of the week in excel extension by @@Mytherin in #11328 * [CI] Fix bash syntax in TwineUpload by @@carlopi in #11333 * Fix #11284: avoid adding the same column multiple times to a primary key/ unique constraint name list by @@Mytherin in #11325 * In ColumnData, limit scan to the current count in the column by @@Mytherin in #11329 * Issue #11269: DISTINCT Sorted Aggregates by @@hawkfish in #11321 * [Attach] Fix bug causing sequences to break attaching databases. by @@Tishj in #11327 * Flatten hash vector before combining list hashes by @@lnkuiper in #11340 * Make sniffer more consistent when nullpadding/ignore_errors are on by @@pdet in #11313 * fix(arrow): union buffer count & handle schema errors by @@Mause in #11326 * fix duckdb-r script by @@Tmonster in #11345 * Fix regression_test_runner.py by @@carlopi in #11346 * Issue #11234: IEJoin Scan Reset by @@hawkfish in #11347 * TPC-H: Use BIGINT for ID fields schema where required by the specification by @@szarnyasg in #11341 * Another round of polishing staged releases by @@carlopi in #11342 * CI: Remove issue labeling workflow by @@szarnyasg in #11355 * RE2 upgrade to version 2023-02-01 by @@hannes in #11252 * File System: Add optional_ptr to various calls, and add support for attaching DuckDB files over S3 by @@Mytherin in #11343 * README: Display different logo for light/dark mode by @@szarnyasg in #11366 * Fix bug in duckdb_bind_blob by @@pfarndt in #11368 * Fix OSX CI by @@samansmink in #11379 * Enable clang-tidy on headers and fix all headers to conform to our clang-tidy rules by @@Mytherin in #11376 * Add logical_type to parameters of format_pg_type by @@Flogex in #11369 * Issue #10965: RESPECT IGNORE NULLS by @@hawkfish in #11372 * Fix building issues in WIN32, remove unnecessary modification. by @@kindred77 in #11356 * Zero-initialize aggregate states with destructors immediately after allocating by @@lnkuiper in #11360 * Update README.md by @@jingshi-ant in #11357 * Issue #10885: Negative Window RANGEs by @@hawkfish in #11390 * Issue #11377: Invertible TIMESTAMP_XXX Casts by @@hawkfish in #11392 * Update init.py To export "extract_statements" function by @@oomojola in # 11394 * Internal #1657: Stricter STRUCT Casts by @@hawkfish in #11396 * allow set readonly on attached db by @@stephaniewang526 in #11397 * Give preference to FSSPEC defined FS by @@pdet in #11400 * Default serialize optional_idx, add skip_default option to json_serialize_sql() by @@Maxxen in #11405 * CI: Also label PRs as 'stale' and close them when there's no activity by @@szarnyasg in #11420 * fix(jdbc): 1-index getBytes() by @@Mause in #11421 * Remove redundant default descriptions by @@szarnyasg in #11415 * clang-tidy: enable cppcoreguidelines-pro-type-const-cast by @@Mytherin in # 11414 * clang-tidy: enable cppcoreguidelines-avoid-non-const-global-variables by @@Mytherin in #11424 * Issue #11419: Quantile Order By by @@hawkfish in #11428 * [CSV Sniffer] Give preference to quoted candidates by @@pdet in #11418 * clang-tidy: enable cppcoreguidelines-virtual-class-destructor by @@Mytherin in #11437 * clang-tidy: enable cppcoreguidelines-[interfaces-global-init|slicing| rvalue-reference-param-not-moved] by @@Mytherin in #11435 * Fix #11393 - improve error message when trying to use a lateral join column in a table function that does not support it by @@Mytherin in #11436 * add readonly to duckdb_databases() by @@stephaniewang526 in #11429 * Fix missing opener propagation by @@quentingodeau in #11454 * Fix #11246: Use SetConsoleCP function to set input to UTF8 when reading by @@Mytherin in #11452 * CLI: Add support for ".edit" or "\e" by @@Mytherin in #11447 * Fix VS2022 Preview ClangCl build by @@bodand in #11456 * Remove an unnecessary line from bind_insert.cpp by @@huachaohuang in #11443 * [CI] Skip ccache for R.yml by @@carlopi in #11459 * Improve binding of CTEs by @@kryonix in #11399 * Move BindCreateIndex from Catalog to Binder by @@philippmd in #11402 * [Substrait-ADBC] Fix for substrait plan execution via ADBC by @@pdet in # 11358 * Removing abort() from RE2 again because Google refuses to use exceptions by @@hannes in #11458 * Defer allocation in read_json by @@lnkuiper in #11378 * [ODBC] Add escape character to ParseStringFilter to support Power Query ('table_name' is escaped to 'table_name') by @@guenp in #11432 * Bump to post-portfile change for duckdb_azure by @@carlopi in #11476 * Reduce memory usage of DELETE operations by @@Mytherin in #11470 * Use optional_idx in more places by @@Mytherin in #11466 * Revert "Move BindCreateIndex from Catalog to Binder" by @@Mytherin in #11478 * [Arrow] Throw on invalid STRUCT type by @@Tishj in #11464 * [Dev] Do not use CatalogEntry references inside Dependency objects. by @@Tishj in #11408 * Fix extension builds by @@carlopi in #11486 * [Fix] Throw BinderException for UNNEST expressions in WINDOW expressions by @@taniabogatsch in #11247 * Check for IUTF8 flag defined before setting it by @@patmaddox in #11488 * Fix #11445: correctly detect recursive aliases when using struct unnest by @@Mytherin in #11497 * Fix #11444: avoid using recursion in string -> list parsing by @@Mytherin in #11498 * Add serialization for LogicalCopyDatabase operator by @@Flogex in #11401 * add support in Julia appender for missing and nothing values by @@rdavis120 in #11508 * [Python] Produce datetime.time values when converting TIME columns to Pandas DataFrame by @@Tishj in #11468 * [Fix][ADBC] Implement required ADBCConnectionGetObjects schema by @@joellubi in #11446 * Add support for decimal modulo operation by @@Mytherin in #11506 * Move CompressedMaterialization inside of StatisticsPropagator by @@lnkuiper in #11495 * Bump stale bot version by @@szarnyasg in #11509 * Rework issue workflow by @@Mytherin in #11522 * [RE2] Add includes and remove potential throw from destructor by @@carlopi in #11513 * Issue #11292: Nested Boolean Compares by @@hawkfish in #11496 * [Dev] Initialize new buffers with garbage data if DESTROY_UNPINNED_BLOCKS is set by @@Tishj in #11270 * Fix timeout in async workflow by @@samansmink in #11525 * Move assertion in json_scan.cpp by @@lnkuiper in #11530 * Issue #11518: TryParseTime by @@hawkfish in #11519 * Fuzzer Bugfixes by @@Maxxen in #11544 * [CI] Fix CI failure on C Enum Integrity Check by @@Tishj in #11547 * [ICU] Use the correct lookup precedence for TimeZone settings by @@Tishj in #11546 * [CI] Move from default GITHUB_TOKEN to specific one by @@carlopi in #11556 * [CI] Fix Deploy step to execute only for duckdb organization by @@carlopi in #11553 * Rework RadixPartitionHashTable task assignment in source phase by @@lnkuiper in #11528 * Run new micro benchmarks in CI when they are added by @@Tmonster in #11532 * Rework vector_hash for ARRAYs by @@Maxxen in #11558 * [Dev] Add assertions around Uncompressed String storage by @@Tishj in #11267 * python: Add missing global options to write_csv by @@jzavala-gonzalez in # 10382 * [Python] Fix issue with lists containing dictionaries of different sizes by @@Tishj in #11095 * [Dev][Python] Add nightly test to execute all sqllogic tests using the python package by @@Tishj in #11137 * Parquet Writer: Early out creating dictionary by @@lnkuiper in #11461 * ODBC driver should ignore "driver" and "trusted_connection" keywords in connection string by @@guenp in #11382 * [ODBC] Fix: Support loading UTF-8 encoded data with Power BI by @@guenp in # 11423 * Draft permissions - bot does not have permission for drafting by @@Mytherin in #11575 * CI: Remove 'needs reproducible example' when 'reproduced' label is applied by @@szarnyasg in #11576 * Various fixes & clean-up around STRUCT UNNEST by @@Mytherin in #11580 * Update token by @@Mytherin in #11592 * Update issue template by @@szarnyasg in #11577 * [CI] Remove GITHUB_PAT variable from R-CMD-check by @@carlopi in #11593 * Respect read-only mode in dbgen and dsdgen by @@Mytherin in #11585 * Bump-back duckdb_azure to pre-lzma custom vcpkg-port by @@carlopi in #11595 * Correctly handle database names with quotes in USE statement by @@Mytherin in #11587 * Bump postgres version and build arrow also for windows by @@carlopi in # 11604 * Support reading gzipped files in the test runner by @@chrisiou in #11600 * initializes unknown indexes on catalog lookup by @@Maxxen in #11551 * Fix Progress Bar for many large CSV Files + Adjustment to not store buffers from compressed files over single threaded scans by @@pdet in #11273 * CSV Rejects Tables 2.0 by @@pdet in #11512 * Fix topn placement by @@Tmonster in #11601 * Fix various issues found by oss-fuzz by @@Mytherin in #11613 * [ODBC] Fix: timestamps and times are parsed as dates by Power Query by @@guenp in #11610 * Fix various fuzzer issues, move fuzzer scripts into this repo, and expand reduce_sql_statement to improve test case reduction capabilities of fuzzer by @@Mytherin in #11622 * [Dev] Make the extension_entries.hpp generation script more modular by @@Tishj in #11623 * [Fix][ADBC] Don't filter system catalogs/schemas in ConnectionGetObjects by @@joellubi in #11618 * Add pyodide wheel building github action by @@cpcloud in #11531 * Move away from dynamic_cast to Cast<> infrastructure by @@carlopi in #11619 * Extension Metadata by @@carlopi in #11515 * [Dev] Regenerate query string for IndexCatalogEntry. by @@Tishj in #11462 * Upload pyodide by @@carlopi in #11626 * Add docker alpine build to check on builds by @@carlopi in #11490 * Add Vector Similarity Search (VSS) Extension by @@Maxxen in #11614 * Metadata fix by @@carlopi in #11629 * Fix extension config for arrow, remove patch from sqlite by @@carlopi in # 11628 * [CSV Reader] Resets the buffer manager over recursive scans by @@pdet in # 11631 * Make path to append_metadata.cmake relative to top-level CMakeLists.txt by @@Flogex in #11635 * [CSV Reader] Fixes an issue with conflicting strategies for buffer cleaning by @@pdet in #11630 * Fix more issues found by the fuzzer, extend SQL reduction further by @@Mytherin in #11642 * fix(jdbc): support non-string parameter types by @@Mause in #11646 * Few more fuzzer fixes by @@Mytherin in #11648 * Bump spatial by @@Maxxen in #11650 * Avoid performing Apple codesign on extensions by @@carlopi in #11652 * Filter out single relation predicates before join ordering by @@wangxiaoying in #11645 * Fix last_value in the duckdb_sequences metadata function by @@Tishj in # 11465 * Limit batch insert threads based on available memory, similar to Parquet write by @@Mytherin in #11655 * [Vacuum] Fix serialization and Copy of the VacuumStatement by @@Tishj in # 11656 * More index initialization by @@Maxxen in #11659 * Skip tests with the unzip keyword in python and disable unzip.test for 32bit systems by @@chrisiou in #11658 * Bump extension versions, remove patches by @@carlopi in #11662 * Accept a list of multiple nullstring values for CSV Files by @@pdet in # 11616 * Include falloc to fix build on some Linux systems by @@zmbc in #11663 * Fix #11469 - make unnest parameters case-insensitive by @@Mytherin in #11667 * Fix #11467: correctly merge unnamed structs and structs in CombineEqualTypes by @@Mytherin in #11668 * Skip ADBC tests if python version is not 3.9 or higher by @@pdet in #11653 * Fix #11621 - correctly zero-initialize padding bits in bitpacking compression by @@Mytherin in #11671 * Fix #11542 - correctly check if a column data segment has updates, and clean up the updates by @@Mytherin in #11670 * Make UNION BY NAME also use ForceMaxLogicalType, similar to UNION by @@Mytherin in #11665 * Fix extension_version propagation for external extensions by @@carlopi in # 11672 * Allow decimal type in CSV auto_type_candidates option by @@pdet in #11675 * Fix #11484: support constant indexes in ARRAY - e.g. ARRAY(SELECT .. ORDER BY 1) by @@Mytherin in #11674 * Improve hive type auto-casting so that it looks at all files instead of only the first file by @@Mytherin in #11676 * Fix #11669: deduplicate column names in pivot correctly by @@Mytherin in # 11678 * Disable setting console pages by default, and add .utf8 setting by @@Mytherin in #11682 * bump vss, handle reverting append when index is unknown by @@Maxxen in # 11681 v0.10.1 Bugfix Release What's Changed * Remove visualizer leftovers by @@Y-- in #10642 * Add explicit numbering to C enums + various compilation/CI fixes by @@Mytherin in #10649 * Disable print method for CSV scanner for R build by @@hannes in #10650 * Fix #10548 for the DUCKDB_NO_THREADS case by @@carlopi in #10654 * Allow StorageExtension to extend DuckCatalog implementation in order to integration with observability system by @@bleskes in #10643 * Update storage info for v0.10.0 by @@szarnyasg in #10660 * Revamp duckdb-wasm extensions CI by @@carlopi in #10672 * [CI] Re-enable skipped test window-rows-overflow.test by @@Tishj in #10679 * Catch: prominently display skipped tests by @@Mytherin in #10669 * Update Julia to 0.10.0 by @@Mytherin in #10689 * Ingestion benchmark framework by @@Tmonster in #10341 * [ICU] Add casts from Timestamp_* to TimestampTZ by @@Tishj in #9539 * DISTINCT ON - greatly improve performance by rewriting ordered FIRST aggregate into arg_min_null by @@Mytherin in #10684 * Fix #10685 - support aliases in join clause by @@Mytherin in #10691 * Use assertThrows for throwing assertions in JDBC tests by @@peteraisher in # 10448 * Casts: report error location in query for failed casts by @@Mytherin in # 10694 * Fix duckdb spelling in _extension_deploy.yml by @@carlopi in #10717 * Fuzzer #1374: ARG_XXX By Decimal by @@hawkfish in #10728 * [Python] Rework the python regression test script by @@Tishj in #10715 * Removes static member string by @@TinyTinni in #10733 * Fuzzer #1372: Order Bind Failure by @@hawkfish in #10727 * Fuzzer #1380: To Weeks Overflow by @@hawkfish in #10726 * Various fixes by @@carlopi in #10708 * Unittest does not satify assertion on MSVC/Debug by @@TinyTinni in #10738 * Fix OrderPreservationType issue of MATERIALIZED CTEs by @@kryonix in #10587 * Map creation fixes and refactoring by @@taniabogatsch in #10436 * Fuzzer #1383: NULL Range Arguments by @@hawkfish in #10723 * Fuzzer #1382: Window Stats Overflow by @@hawkfish in #10725 * Comment on view columns by @@samansmink in #10710 * Union exclude by @@Tmonster in #10688 * move the logic for immediate_transaction_mode to the physical operator by @@peterboncz in #10739 * [C API] Small fix and more tests by @@taniabogatsch in #10748 * List_slice bug fix by @@maiadegraaf in #10747 * Enable azure autoload by @@samansmink in #10746 * Parquet writer - reduce memory usage of order-preserving write by @@Mytherin in #10756 * Refactor csv reader includes because of r path length limitations by @@hannes in #10658 * Arrow String View Type by @@pdet in #10481 * local_file_system.cpp: minor fix for macOS libproc code by @@barracuda156 in #10758 * Make unnamed_subquery naming predictable by @@Mytherin in #10765 * [Python] Fix overflow issue in PandasAnalyzer by @@Tishj in #10768 * Throw when trying to consume over 128 byte decimals by @@pdet in #10601 * CLI: Right-align numerics in markdown tables by @@Mytherin in #10767 * Fuzzer #1399: Window NULL RANGE by @@hawkfish in #10776 * [CSV Sniffer] Minor sniffer tweak to give preference to dialects that generate the least errors if ignore_errors = true by @@pdet in #10777 * Add large benchmark directory by @@Tmonster in #10763 * Improve UNPIVOT error messages, and allow expressions in unpivot by @@Mytherin in #10773 * Add a method UUID::FromUHugeint to generate a UUID from a uhugeint_t by @@Mytherin in #10771 * [CSV Reader] Add lock to buffer reset by @@pdet in #10791 * [WINDOWS] Add "/bigobj" that solves compile issue during debug by @@maiadegraaf in #10782 * fix: not over-call AllSecrets by @@stephaniewang526 in #10807 * Update readme by @@szarnyasg in #10814 * [ODBC] Rework Connect to the ODBC driver and add functionality to set all DuckDB configurations in the Connection String by @@maiadegraaf in #10692 * Fix arrow conversion, map doesn't support large offset by @@yiyuanliu in # 10796 * [CSV Reader] Spinlock over GetLine Error + New Strategy for dialect candidates by @@pdet in #10755 * Trivial fixes by @@carlopi in #10816 * Fix unicode handling in underscore of LIKE operator by @@Mytherin in #10821 * JDBC spurious CI failure - an exception being thrown in this test is a race condition by @@Mytherin in #10825 * Benchmark runner - allow files (e.g. CSV/Parquet) to be cached using the cache command by @@Mytherin in #10817 * Fix #10803 - correctly reclaim space of list indexes when columns are dropped by @@Mytherin in #10822 * [Upsert] INSERT OR REPLACE fixes by @@Tishj in #10789 * [Dev] Add an optional time out in seconds to run_tests_one_by_one.py by @@Tishj in #10744 * Maintain names in COLUMNS(*) expression, and allow aliasing multiple columns using {column} by @@Mytherin in #10774 * Disable AWS/Azure on Windows for now by @@Mytherin in #10827 * [Dev] Bump memory limit on batch_memory_usage.test_slow by @@carlopi in # 10845 * Fix coverity apt-get by @@carlopi in #10838 * minor: FixedSizeBuffer::Pin move shared_ptr rather than copying by @@mapleFU in #10837 * ci: Upgrade workflows to actions/setup-python@@v5 by @@krlmlr in #10832 * Fuzzer #1389: ARG_XXX Decimal Casts by @@hawkfish in #10742 * Contributor guide: Fix new issue link by @@szarnyasg in #10836 * Changing source to src in relational_constraints query by @@Dtenwolde in # 10848 * Fix: correctly calculate the range of build side for perfect hash join by @@gitccl in #10446 * [Python] Fix issue caused by deadlock between thread.join() and acquiring the GIL by @@Tishj in #10854 * [CSV Parser] 8-Byte Skipping instead of 1-Byte when possible by @@pdet in # 10855 * Add components of the version to duckdb.hpp by @@ahuarte47 in #10840 * [CSV Sniffer] Tweaking header detection by @@pdet in #10714 * Check if directory exists before removing files in regression test runner by @@Tmonster in #10859 * [Extension] Add CatalogType to the list of functions generated in extension_entries.hpp by @@Tishj in #10597 * Regression test build side probe side by @@Tmonster in #10585 * [Arrow] Fix issue surrounding lifetime of dictionary arrays by @@Tishj in # 10610 * Fix #10745 - correctly deal with empty float columns in floating point compression routines by @@Mytherin in #10863 * [Extensions] Build fixes by @@carlopi in #10860 * Fix MSVC linking issue with workaround by @@samansmink in #10865 * Reduce memory usage & avoid spilling to disk unnecessarily for order-preserving table creation/insertion by @@Mytherin in #10862 * pb/avoid GetSchema opening a transaction by @@peterboncz in #10740 * Support dollar-quoted string-constants in the CLI by @@Mytherin in #10879 * Shell: avoid printing "Error: " prefix if the error message already has a prefix (e.g. Binder Error:, Parser Error:, etc) by @@Mytherin in #10880 * Partially fix #10751: correctly catch exceptions in sqlite3_print_duckbox by @@Mytherin in #10881 * Reset refresh in CompressedFile::Close() by @@Maxxen in #10882 * [CSV Reader] Lock when getting progress by @@pdet in #10884 * [CSV Sniffer] Early out if things go wrong in dialect detection by @@pdet in #10872 * [CSV Reader] Fix for skipping mix of newline delimiters by @@pdet in #10864 * Parallelize format.py script by @@hatvik in #10646 * Remove Old PSQLODBC scripts by @@maiadegraaf in #10888 * Add update_odbc_path.py to ODBC bundle by @@maiadegraaf in #10895 * Improve Wasm.yml workflow by @@carlopi in #10899 * [Parquet] Fix #10829, write correct data page offset in the presence of dictionaries by @@hannes in #10890 * Table name binding does not fail for non-existent tables in DROP TABLE statements by @@NiclasHaderer in #10893 * Fix #10889 - correctly deal with compressed vectors in struct filterpushdown of ColumnSegment::FilterSelection by @@Mytherin in #10896 * CI: Disable julia nightly for now by @@Mytherin in #10905 * CLI - add support for rendering errors/matching brackets for square ([]) and curly ({}) brackets as well by @@Mytherin in #10904 * Storage: Fix an internal exception that could be triggered when deleting many rows and checkpointing repeatedly by @@Mytherin in #10897 * LIMIT/OFFSET clean-up by @@Mytherin in #10873 * Add ARRAY to test_all_types + IO and some clients by @@Maxxen in #10850 * Use M1 (ARM) OSX runners by @@hannes in #10670 * build: restore tarball build support by @@Mause in #10900 * Fix #10902 - allow more expressions to be used with an indirection without brackets (. or [], etc) by @@Mytherin in #10909 * feat(jdbc): fixed size array support by @@Mause in #10911 * Add regexp_split_to_table macro by @@szarnyasg in #10898 * [MetaTransaction] Add lock on modifying all_transactions and transactions by @@Tishj in #10799 * Issue #10809: RANGE Hint Corrections by @@hawkfish in #10828 * Enable the progress bar (without printing) in unittests by @@Mytherin in # 10908 * [Python][Dev] Fix issue in read_csv related to the s3 extension by @@Tishj in #10690 * Add support for the C API duckdb_query function to the Julia api by @@rdavis120 in #10886 * Fix #10501 - in LocalFileSystem::Write split writes into batches of at most 2GB by @@Mytherin in #10912 * Correctly reset data chunk in RETURNING of DELETE by @@Mytherin in #10915 * bitstring_agg had a trigger-able assertion, [duckdb-fuzzer/#1414] by @@hannes in #10918 * Shell: Remove IEE754 function from CLI by @@Mytherin in #10919 * Use correct index in string to nested cast error handling by @@Mytherin in # 10920 * Batch memory manager - keep track of all used memory correctly and enforce that unflushed memory is correctly set to 0 when we are finished by @@Mytherin in #10922 * Fix #9975 - correctly open (and keep open) a transaction when checking if prepared statement needs to be rebound by @@Mytherin in #10923 * CLI - Insert spaces when copy-pasting tabs by @@Mytherin in #10924 * feat: exposing ssl ca cert path to httpfs by @@pvaezi in #10704 * Centralize dynamic cast check and disable on MacOS by @@Mytherin in #10925 * Checked Numeric Casts by @@hannes in #10870 * Avoid running numeric cast checks when CRASH_ON_ASSERT is enabled by @@Mytherin in #10942 * Set duckdb_api to 'python jupyter' if in Jupyter notebook by @@guenp in # 10931 * Array fuzzer issue fixes by @@Maxxen in #10944 * Fix assertion trigger in FilterCombiner::AddTransitiveFilters by @@Mytherin in #10941 * Support recursive describe queries (i.e. DESCRIBE(DESCRIBE ..)) by @@Mytherin in #10945 * Avoid throwing null pointer exception in Window Segment Tree destructor by @@Mytherin in #10937 * Fix an issue where partitions were not correctly considered in bound window expression equality by @@Mytherin in #10939 * Fix for limit % with subquery on an empty table by @@Mytherin in #10946 * Correctly visit all expressions during lateral join decorrelation, particularly with nested lateral joins by @@Mytherin in #10936 * when you add the relation, make sure you call gettableIndexes() on th?? by @@Tmonster in #10949 * Internal #1428: Interval Subtract Overflow by @@hawkfish in #10957 * Fuzzer #1445: Trap MAKE_DATE/TIME Overflows by @@hawkfish in #10958 * Python.yml: Revert to macos-latest for OSX workflow by @@carlopi in #10970 * Purge queue refactor by @@taniabogatsch in #10594 * Change time from duckdb_time to duckdb_time_struct in duckdb_time_tz_struct by @@Giorgi in #10933 * Add require block_sizes 262144 on tests reading db files by @@carlopi in # 10974 * [duckdb-fuzzer/#1368] - overflow in bitstring_agg on hugeint & uhugei?? by @@hannes in #10971 * Fix LIST->ARRAY TRY_CAST when list sizes mismatch by @@Maxxen in #10973 * Add a micro extended benchmark. by @@Tmonster in #10943 * [Dev] Move TemporaryFileManager and friends out of StandardBufferManager by @@Tishj in #10938 * [ODBC] Reorganize Directory Structure by @@maiadegraaf in #10979 * Internal #1385: Window Partition Collation by @@hawkfish in #10985 * Fuzzer #1471: Trap MAKE_DATE Overflows by @@hawkfish in #10987 * Range checks for ACOS by @@hannes in #10972 * improve CheckBoundaryValues in TopN by @@xuke-hat in #10955 * CSV tests - use TEST_DIR to prevent leaking file by @@Mytherin in #10991 * Autoload INET and ICU (and add back sqlite and postgres as autoloadable) by @@carlopi in #10948 * Fuzzer #1468: Window RANGE Types by @@hawkfish in #10990 * Fuzzer #1446: Quantile Hugeint Interpolation by @@hawkfish in #10983 * Override git hash / git version by @@carlopi in #10977 * [Storage] Only call FinalizeOptimisticWriter after storage merge has succeeded by @@Mytherin in #10998 * Add MetaTransaction::GetTransaction to threadsan suppressions (false positive) by @@Mytherin in #11001 * Various fixes: CMake + generated extension_entries.hpp checks by @@carlopi in #10994 * Nightly Wasm build fix by @@taniabogatsch in #10993 * Fix return null constant in array_slice and other array issues by @@Maxxen in #10992 * Add correct table bindings for window relations. by @@Tmonster in #10997 * Clean up ExecutorTask and simplify waiting for all tasks to be cancelled by @@Mytherin in #11005 * Various JSON thread sanitizer fixes by @@Mytherin in #11004 * Fix warnings in ALP and logical_insert by @@carlopi in #11008 * Check IsLoaded() before importing cached item by @@Tmonster in #11007 * Issue #10995: ICU VARCHAR TIMETZ by @@hawkfish in #11002 * Fix #10982 - only update total rows of row group collection after we finish appending to prevent other readers from attempting to initialize scans on in-progress appends by @@Mytherin in #11011 * [Nightly] Block size nightly test changes by @@taniabogatsch in #11010 * ATTACH with reserved names (temp/main) by @@Mytherin in #11020 * Fix various tests for vector_size = 2 by @@Mytherin in #11027 * [CI] Create a bigger table in interrupt test by @@Mytherin in #11025 * Issue #10995: TIMETZ DST Fix by @@hawkfish in #11024 * Fixup LinuxRelease.yml release: unittester was not invoked correctly by @@carlopi in #11022 * In DatabaseInstance destructor - destroy TaskScheduler first by @@Mytherin in #11021 * Refactor ATTACH options by @@taniabogatsch in #11016 * Fix #11033: don't reset arena allocator in between calls to streaming window expression by @@Mytherin in #11039 * Avoid checking LastModifiedTime for remote files in object cache by @@Mytherin in #11034 * [Block Size Nightly] Enable more block size nightly tests by @@taniabogatsch in #11036 * Add missing pipeline dependencies in recursive CTE by @@kryonix in #11043 * Use batch limit only when limit + offset are small constants by @@Mytherin in #11035 * Add New CSV Error for Invalid Unicode by @@pdet in #10984 * [FIX] Lambda bug in subqueries by @@taniabogatsch in #11046 * [Swift] performance optimisations by @@tcldr in #11052 * add concat_ws to spark API by @@nicornk in #11051 * feat(jdbc): expose comments via jdbc methods by @@Mause in #11031 * [CI / Tests] Disable CSV sniffer test for smaller vector sizes and reduce block-size nightly runtime by @@taniabogatsch in #11055 * [CSV Sniffer] Consider date/timestamp formats from the user when sniffing by @@pdet in #11057 * Extend the "Contents of view were altered" error with more information by @@Tishj in #11064 * [Python] Add some numeric and string functions to spark API by @@mariotaddeucci in #11067 * [ODBC] Allow multiple statements to be executed using SQLExecDirect by @@maiadegraaf in #11038 * [CSV Reader] Apply projection on over buffer values. by @@pdet in #11056 * Python: use short paths for Windows by @@Mytherin in #11068 * Fix #10752: Add support for Parquet encryption on Windows by @@Mytherin in # 11069 * [Python] Code Quality - PEP8 Compliant + only relevant imports by @@mariotaddeucci in #11070 * [CI] Add patch argument to patch the extension's sources before building by @@krlmlr in #10831 * fix(jdbc): support fractional seconds in getTime by @@Mause in #10707 * Fix #11071 - correctly report progress when scanning multiple Parquet files by @@Mytherin in #11072 * Add ipv6 inet + minor fixes by @@carlopi in #11073 * Implement IPv6 support in the inet extension. by @@troycurtisjr in #10839 * [CI] Add step to verify C API enum integrity. by @@Tishj in #10664 * Issue #10995: TIMETZ DST Fix by @@hawkfish in #11079 * Check Nested Types for UTF-8 Correctness by @@pdet in #11086 * Add scope column to duckdb_settings by @@Tishj in #11017 * Partitioned write - flush batches periodically (every 500K rows) instead of only writing when all data has been gathered by @@Mytherin in #10976 * [Python] Add extract_statements and the Statement class by @@Tishj in #10891 * [Python] Improve performance of conversion to Numpy/Pandas for nested lists by @@Tishj in #10826 * Fix an InternalException caused by DICTIONARY_VECTOR inside map_from_entries by @@Tishj in #11091 * Fix #11084 - fixes an issue with the Parquet writer when writing vectors of lists with repeated list elements (as can be generated through a join) by @@Mytherin in #11094 * Add callbacks for newly added connections, and allow extensions to rebind queries as a result of planning failures by @@Mytherin in #11096 * Fuzzer #2376: INTERVAL Muliply Overflow by @@hawkfish in #11100 * Make test/sql/copy/csv/test_limit_spinlock.test a slowtest by @@pdet in # 11088 * Fix #11063 - avoid throwing exception in InClauseRewriter by @@Mytherin in # 11090 * Add a hint on how to resolve lockups when using ninja. by @@troycurtisjr in #11074 * Fix LocalFileSystem::Read/Write, update location after read/write some data by @@yiyuanliu in #11105 * run_tests_one_by_one - add a default timeout of 1 hour by @@Mytherin in # 11104 * [Fix] Aliases in subqueries by @@taniabogatsch in #11103 * [CSV Scan] Implement ignore_erros for Dates/Timestamps/Decimals by @@pdet in #11083 * Fix TaskScheduler deadlock on NumberOfThreads by @@Tishj in #11093 * Fix return null constant in list_resize and list_aggr by @@maiadegraaf in # 11111 * Add rowsort to more tests for queries that don't have a defined order by @@Mytherin in #11110 * Do not extract filters that cannot be hyper edges (Join Order Optimizer) by @@Tmonster in #11108 * Add Dictionary vector verification by @@Mytherin in #11114 * [Dev][Python] Make test_httpfs.py error test more lenient by @@Tishj in # 11125 * Fix ConstantVector::Reference for dictionary arrays by @@Maxxen in #11136 * Disable jemalloc for ARM distributions and clean up when closing DB by @@lnkuiper in #11130 * Case senstivity issue secret manager by @@samansmink in #11128 * Bump az aw vcpkg by @@samansmink in #11127 * Fix broken micro benchmarks so they can be run weekly by @@Tmonster in # 11113 * Refactor OSX.yml, now inputs can be provided by @@carlopi in #11133 * Merge main into feature by @@Tishj in #11141 * Revert "Merge main into feature" by @@Mytherin in #11145 * Fix upload assets script by @@carlopi in #11144 * Bump spatial by @@Maxxen in #11132 * Fix upload assets OSX/2 by @@carlopi in #11148 * Add more vector type verification tests/settings by @@Mytherin in #11138 * Allow for customization of catalog lookup behavior for different catalog types by @@Mytherin in #11151 * [Python][Arrow] Don't deduplicate column names when outputting to Arrow by @@Tishj in #11160 * More Array and Union fixes by @@Maxxen in #11161 * Refactor upload logic (towards staged releases) by @@carlopi in #11156 * Fuzzer #2490: Generate NULL TIMESTAMPTZ by @@hawkfish in #11143 * Add folder parameter to upload logic and upload also twine artifacts by @@carlopi in #11169 * CI: Find mirror issues among all issues, not just open issues by @@szarnyasg in #11170 * Fix TupleDataCollection serialization of dictionary vectors containing nested data by @@lnkuiper in #11174 * Allow overriding of git describe also in scripts (via OVERRIDE_GIT_DESCRIBE environment variable) by @@carlopi in #11179 * Python staged releases: centralized staged upload by @@carlopi in #11187 * Fix RevertAppendInternal by @@Mytherin in #11177 * TwineUpload: Add awscli dependency + minor rework by @@carlopi in #11193 * Fix issue in copy constructor of ExtraDropSecretInfo by @@samansmink in # 11190 * Unify CSV/JSON and Parquet Batch Writing Code - and fix memory management issues in CSV/JSON writing by @@Mytherin in #11188 * More conservative dummy list entry estimation by @@Maxxen in #11185 * retry on 500 error by @@samansmink in #11184 * Fix warning on unused utf_type by @@carlopi in #11198 * Remove outdated duckdb-node related scripts by @@carlopi in #11180 * Add StagedUpload.yml by @@carlopi in #11189 * Improving CSV Casting error message by @@pdet in #11183 * Improve conversion error message in Parquet reader by @@Mytherin in #11199 * Fix thread sanitizer issues by @@Mytherin in #11200 * Internal #1564: Range Join DISTINCT by @@hawkfish in #11205 * small fix secret autoloading, bump azure by @@samansmink in #11182 * CSV reader - suggest enabling null_padding and ignore_errors in case of missing columns by @@Mytherin in #11201 * CI: Create/label mirror issue job should list all internal issues by @@szarnyasg in #11204 * [Python] Add IS NULL / IS NOT NULL support to Expression API by @@cmdlineluser in #11175 * Remove old assertions in SegmentTree by @@Mytherin in #11208 * [lambda] Fix for list_reduce giving the wrong result by @@maiadegraaf in # 11171 * [Dev] Fix various issues discovered by #11137 by @@Tishj in #11210 * [Dev] Fix py override describe by @@carlopi in #11209 * Sanitize CSV Newline identifier for writing CSV files by @@pdet in #11106 * Fix persistent secret file permissions by @@samansmink in #11172 * Retry Binding Prior To Execution by @@Mytherin in #11149 * Avoid copying LogicalType in FlatVector::SetNull. by @@yiyuanliu in #11214 * Review of CI on tags + add R extensions CI to InvokeCI.yml by @@carlopi in # 11212 * Fix python and apply patches + bump extensions by @@carlopi in #11217 * Disable jemalloc on arm in Python package as well by @@Mytherin in #11218 DuckDB 0.10.0 "Fusca" What's Changed * feat(jdbc): support uuid param type by @@Mause in #9164 * Bump ADBC to v0.7 by @@pdet in #9185 * Add support for parquet key-value metadata by @@Maxxen in #9126 * Default to JSON type if objects have an inconsistent structure by @@lnkuiper in #9086 * Add schema parameter to read_parquet by @@lnkuiper in #9123 * [Python] Add the ability to provide a list of files to read_csv by @@Tishj in #8977 * Issue #7672: TIMESTAMP_XX to DATE by @@hawkfish in #9279 * N-ary lambdas, and indexes as lambda parameters by @@taniabogatsch in #8851 * Feature: Fixed size list nested type (ARRAY) by @@Maxxen in #8983 * Fix unused warning by @@Maxxen in #9318 * Internal #215: Window EXCLUDE Functionality by @@hawkfish in #9220 * Add json_serialize_plan, json_serialize_sql tweaks by @@Maxxen in #9230 * Add create statement support to fuzzer by @@Tmonster in #9011 * Fix for issue #8108: Random() in lambda by @@maiadegraaf in #9353 * Fix Lambda Serialization by @@taniabogatsch in #9323 * Allow file_row_number with parquet schema option by @@samansmink in #9290 * CSV - Always run sniffer by default by @@pdet in #9250 * [Python Dev] Import items lazily by @@Tishj in #8741 * Array fixes + make validity more lazy by @@Maxxen in #9400 * Lambda performance revamp by @@taniabogatsch in #9395 * [Python] Support replacement scan on connection.table() method by @@Tishj in #9427 * [Dev] Fix failure in Format Check CI job by @@carlopi in #9516 * Fix parquet serialization by @@Maxxen in #9485 * Support gcs:// and r2:// URLs to read data from GCS and R2 by @@chrisiou in #9388 * Don't reset validity target count by @@Maxxen in #9565 * Merge into feature by @@carlopi in #9583 * Additional list functions by @@cryoEncryp in #8907 * [Python] Support Optional[...] in DuckDBPyType by @@Tishj in #8658 * C-API: support streaming arrow query by @@Virgiel in #8642 * Add ToString and Print method for JoinRelationSetManager and Fix JoinNode Print by @@Light-City in #9040 * Removed artificial HUGEINT minimum by @@nickgerrets in #9441 * Parquet Encryption by @@lnkuiper in #9392 * Internal #330: Quantile Performance by @@hawkfish in #9461 * CAPI: Make it possible to create enum types by @@alnkesq in #8788 * 5614 database invalidated by @@Tmonster in #9513 * Add support for proper scoping (catalog + schema) to custom types by @@Mytherin in #9622 * Internal #576: strptime strftime infinities by @@hawkfish in #9615 * ATTACH IF NOT EXISTS by @@Mytherin in #9627 * Small benchmark changes by @@taniabogatsch in #9638 * add option for keep_alive setting by @@samansmink in #9648 * Add "getenv" function to shell which can be used to read environment variables, and allow functions to be used in SET statements/PRAGMA statements by @@Mytherin in #9651 * Julia release by @@Mytherin in #9670 * In set operations ORDER BY columns refer to the first set operation in SQL - so the reference is not ambiguous by @@Mytherin in #9658 * Replace old logos by @@szarnyasg in #9674 * Fix dbgen/dsdgen when using custom catalog and schema by @@ywelsch in #9686 * [Arrow] Properly use the parent's array.offset in many places in the scan by @@Tishj in #9661 * Update issue template with API/extension repositories by @@szarnyasg in # 9693 * [Python] Fix lossy datetime.timedelta to INTERVAL conversion by @@Tishj in # 9688 * Issue template: Report vulnerabilities via dedicated channel by @@szarnyasg in #9711 * Fix #9601: Call correct method in duckdb_pending_execution_is_finished by @@Mytherin in #9728 * Merge Feature Into Main by @@Mytherin in #9726 * feat(c): add functions for determining statement/return types by @@Mause in #9430 * Issue #9673: ICU DST Truncation by @@hawkfish in #9705 * Deserialize header fields by @@carlopi in #9687 * fix: restore support for windows network paths by @@Mause in #9725 * Constant time attach path lookup and locking to ensure unique file handles by @@taniabogatsch in #9671 * [ART] WAL serialization, automatic checkpointing, decoupling catalog and storage, index names by @@taniabogatsch in #9339 * Fix expanding structs in queries with ORDER BY by @@taniabogatsch in #9481 * Only emit batch indices valid within the current pipeline by @@ywelsch in # 9702 * Fix #9717: Correctly set null statistics of children of structs by @@Mytherin in #9733 * Fixes to warning and rendering of bar() by @@carlopi in #9734 * Don't show "blabla" as part of syntax error by @@mlafeldt in #9746 * Fix #9742: correctly catch empty ROW case in UPDATE by @@Mytherin in #9744 * Explicitly attach duckdb file type by @@taniabogatsch in #9747 * For ATTACH - Resolve extension prefix before determining the name so "sqlite:file.db" is again correctly aliased as "file" instead of "sqlite:file" by @@Mytherin in #9753 * Wasm: Add wasm_threads as a class of built extensions by @@carlopi in #9735 * Remove index joins by @@taniabogatsch in #9751 * Support EXCEPT ALL and INTERSECT ALL by @@hannes in #9636 * Add nightly deploy script by @@samansmink in #9761 * update vcpkg by @@samansmink in #9759 * Close s3 filehandle on destruction by @@samansmink in #9758 * add list of collations that are required to determine equality. by @@Tmonster in #9757 * Apply fix for patching vcpkg in extension workflow by @@samansmink in #9767 * Internal #766: SkipList Coin Toss by @@hawkfish in #9772 * Support reading large decimals into doubles in the Parquet reader by @@Mytherin in #9770 * ATTACH - Always run ExtractExtensionPrefix also if a name is provided by @@Mytherin in #9771 * Issue #9762: Interval Fractional Seconds by @@hawkfish in #9773 * Internal #716: Summarize approx_unique BIGINT by @@hawkfish in #9774 * Issue #9755: TIMESTAMP_XX DOUBLE Parts by @@hawkfish in #9769 * Add support for COPY FROM DATABASE statement by @@Mytherin in #9765 * chore: move links duckdblabs -> duckdb by @@dpprdan in #9779 * Detect FreeBSD platform by @@szarnyasg in #9782 * Extention template: Enable DuckDB-Wasm extensions by @@carlopi in #9356 * Version: add info on v0.9.2 by @@carlopi in #9788 * [JDBC] Sync all methods from a statement that interact with a query result by @@pdet in #9659 * format_bytes rework, moving from decimal multipliers to binary ones by @@carlopi in #9736 * Make FORCE CHECKPOINT abort transactions of concurrently running queries by @@Mytherin in #9790 * Bugfix/9768 by @@nbc in #9791 * Cleanup raw pointers from transaction manager by @@Mytherin in #9237 * Partially fix #4182 - write distinct stats for string dictionary columns by @@Mytherin in #9518 * fix(python): minimal changes to support compiling for python 3.12 by @@Mause in #9797 * Implement #2534 - add parquet_file_metadata function that supports scanning top-level file metadata by @@Mytherin in #9793 * Correctly clean up database path when an error is thrown in attach by @@Mytherin in #9792 * Fix cotangent(0.0): should also throw OutOfRange by @@carlopi in #9799 * Call BindSchemaOrCatalog when binding functions so that we can qualify functions with only a database as well by @@Mytherin in #9810 * Fix #9739 - UNIQUE USING INDEX is not supported by @@Mytherin in #9809 * Fix #8596 - use ConstructConstantFromExpression for PIVOT IN list by @@Mytherin in #9813 * Fix #8500 - if we encounter any ambiguity while binding a function with a parameter we rebind during execution by @@Mytherin in #9814 * Remove watchOS from CI tests by @@carlopi in #9820 * Fix #9262 - avoid checking exclusion/replace list when extracting table names by @@Mytherin in #9812 * LIST to VARCHAR cast fix by @@taniabogatsch in #9819 * Fix #9806 - when an overflow is detected during filter pushdown the pushdown should be halted, instead of claiming the result is always false by @@Mytherin in #9816 * Parquet: Include column key-value metadata in parquet_metadata function by @@Mytherin in #9798 * Add duckdb_optimizers function by @@Mytherin in #9815 * Internal #805: Summarize NULL Percentage by @@hawkfish in #9822 * Add s3a s3n protocols for httpfs by @@samansmink in #9817 * S3: Add more details to error on multipart upload by @@carlopi in #9826 * CI fixes by @@Mytherin in #9827 * [Python] Pandas Analyzer no longer trips up when the pandas_analyze_sample would only let it find nulls. by @@Tishj in #9811 * Create streaming result from a prepared statement. by @@Giorgi in #9802 * Clear all updates during checkpoint by @@Mytherin in #9829 * Fix #9825 - disable adding transitive filters for <> in filter combiner by @@Mytherin in #9832 * [Upsert] Do not require ON CONFLICT clause on INSERT OR REPLACE in some situations. by @@Tishj in #9730 * Issue #9631: Time/Interval Hours by @@hawkfish in #9643 * feat(c): support creating nested values in C API by @@Mause in #9438 * For EXPORT DATABASE - always write forward slashes in COPY statements by @@Mytherin in #9833 * Make checking the database path atomic again by turning db_paths into a bloom filter instead of a source of truth by @@Mytherin in #9828 * Issue #9785: Missing Interval Parts by @@hawkfish in #9796 * Issue#604 Support collations during IN/NOT IN operations. by @@StarveZhou in #9724 * Reset expression before returning binder error by @@taniabogatsch in #9821 * Fix: Undefined behaviour in Bitpacking compression by @@nickgerrets in #9844 * Add ORDER BY to List_Select test by @@maiadegraaf in #9847 * CSV Sniffer Function by @@pdet in #9522 * Expand Progress API by @@pdet in #9531 * Fix #8095: Adjust the LG_PAGE parameter in jemalloc to accommodate the 64KB PAGE SIZE in aarch64 Linux systems. by @@vincent-chang in #9642 * Improve error message for index limitations by @@szarnyasg in #9862 * Fix empty box in explain analyze statement by @@chrisiou in #9860 * [Python] No longer scan datetime.datetime.max as infinity by @@Tishj in # 9848 * Move Wasm logic to inside CMake by @@carlopi in #9830 * Fix s3fs close issue by @@samansmink in #9846 * Fix TSAN issue related to db_paths_lock by @@Mytherin in #9868 * Fix #9863 - avoid moving constants for DATE - DATE subtractions by @@Mytherin in #9874 * Disable progress bar test by @@Mytherin in #9875 * add FROM to InitialKeywords vector in autocomplete extension by @@hamilton in #9877 * [Optimizer] regexp_matches fix InternalException caused by NULL by @@Tishj in #9872 * [ADBC Test] Prevent segfault in test by @@Tishj in #9871 * Fix progress test by properly initialising variables by @@pdet in #9882 * Capitalize URL in httpfs extension flags by @@szarnyasg in #9884 * Mark BufferPool getters const by @@Y-- in #9885 * make BufferPool members protected by @@jkub in #9886 * Parquet: Support more physical types of time columns with time zone by @@hannes in #9895 * docs(capi): add tests for prepared statement streaming by @@Mause in #9892 * Fix deadlock in LockClients when checkpointing multiple databases, and avoid locking all clients for regular checkpoints by @@Mytherin in #9898 * Disable copy constructor of connection by @@Mytherin in #9899 * Add TableCatalogEntry to bind info by @@Mytherin in #9896 * Issue #9869: Strptime Week Start by @@hawkfish in #9890 * Fix #9867 - correctly propagate relation name in COLUMNS expression by @@Mytherin in #9904 * Fix progress bar (again) by @@pdet in #9905 * Safeguard uses of ColumnDefinition::DefaultValue by @@Tishj in #9842 * Various CI fixes by @@Mytherin in #9903 * [ADBC] Add support for windows. by @@pdet in #9357 * Fix #8905 - make duckdb_rows_changed work with both new and deprecated results by @@Mytherin in #9907 * ART duckdb versions test requires 64-bit system by @@taniabogatsch in #9906 * Internal #751: Shared Window Partition by @@hawkfish in #9839 * Issue #9887: ISO Format Directives by @@hawkfish in #9910 * Fix: string to integer cast by @@nickgerrets in #9581 * [Test] Make test_progress_bar.cpp output more verbose by @@Tishj in #9916 * Refactor and fix ART concurrency tests to avoid spurious CI failures by @@taniabogatsch in #9919 * Write old (empty) index_pointers to table metadata for forwards compatibility with v0.9.2 by @@Mytherin in #9923 * User agent in http header by @@elefeint in #9632 * Add support for vacuuming partial deletes during CHECKPOINT by @@Mytherin in #9931 * JDBC: DuckDBNative - Close Shared Object InputStream Properly by @@brianwyka in #9933 * Fix ATTACH of foreign key by @@taniabogatsch in #9932 * [DependencyManager] Rework internals of the DependencyManager by @@Tishj in #9715 * Internal #873: Empty Aggregate Frames by @@hawkfish in #9936 * Add support for right_semi and right_anti. PR 2 by @@Tmonster in #9897 * httpfs: fix null pointer dereference in AWSEnvironmentCredentialsProvider by @@mlafeldt in #9953 * Limit initial combine capacity in RadixPartitionedHashTable by @@lnkuiper in #9946 * [Python][Dev] Switch to using pyproject.toml when building and running CI. by @@Tishj in #9944 * Issue#9795 min/max doesn't use collation by @@StarveZhou in #9855 * [Arrow][UDF] Properly support side_effects parameter for Arrow UDFs. by @@Tishj in #9959 * JDBC - Appender for Decimal by @@Jens-H in #9568 * [Fix] No duplicates in list_intersect by @@taniabogatsch in #9947 * [Python] Fix the ability to provide pandas_analyze_sample_size in the config dictionary to connect by @@Tishj in #9961 * [SQLLogicTest] Expected error message is no longer optional for statement error by @@Tishj in #9962 * fix offset type in list_casts and array_casts. by @@yiyuanliu in #9972 * Issue #9978: Approximate Quantile Overflow by @@hawkfish in #9985 * fix regexp_replace bug by @@alitrack in #9938 * chore: Remove dead code by @@krlmlr in #9992 * Internal #898: Totally Ordered Intervals by @@hawkfish in #9994 * Swift Readme.md small fix by @@atacan in #9977 * Issue #9956: Alternative TIME Casts by @@hawkfish in #9984 * Make IEJoin code more clear by @@xuke-hat in #9973 * Parallel Checkpointing by @@Mytherin in #9999 * [Fix] ambiguous lambda parameters by @@taniabogatsch in #9974 * [Python][Dev] Disable test causing issues on Python3.7 by @@Tishj in #10005 * Shell: Many improvements/fixes to multi-line mode, enable multi-line mode by default by @@Mytherin in #10015 * Increase CLI history's max length to 1000 by @@szarnyasg in #10006 * [Fix] nullptr dereference when analyzing nested types by @@taniabogatsch in #10004 * Improving error message when trying to open a locked database by @@hannes in #9988 * fix(function): fix ceil function by @@sundy-li in #10014 * fix read parquet progress and read csv progress. by @@yiyuanliu in #10013 * [Python][StreamQueryResult] Fix memory ownership issues in StreamQueryResult::FetchRaw by @@Tishj in #9968 * Revert to old method of computing terminal size as new method does not play nice with lldb by @@Mytherin in #10026 * Issue #9762: Interval Fractional Parts by @@hawkfish in #9954 * Fix #9380 and #9738 by @@lnkuiper in #9990 * Fix array subquery by @@chenzl25 in #10025 * Issue #4545: Windowed Distinct Aggregates by @@hawkfish in #9754 * add rowsort to test by @@stephaniewang526 in #10036 * The UHUGEINT type by @@nickgerrets in #8635 * build(python): fix python pkg version by @@Mause in #10049 * Add support for FixedPointDeclimals v0.5 by @@mcmcgrath13 in #10039 * DuckDB Secrets by @@samansmink in #10042 * Improve progress bar for Aggregation and limit threads for large data sizes by @@lnkuiper in #9989 * Internal #940: GCC Window Distinct by @@hawkfish in #10054 * Fix #10058: correctly handle unicode literals in regexp optimizer by @@Mytherin in #10061 * Update database_size.hpp by @@rjatwal in #10072 * enable overriding per-query working-memory target. by @@jkub in #9544 * fix(c): fix duckdb_create_union_type by @@Mause in #10097 * Call InitialCleanup in PendingQuery(SQLStatement) by @@ywelsch in #10083 * Removing useless variables by @@ywgrit in #10082 * [WIP] Support dot notation for JSON by @@ankrgyl in #9499 * Refactor deploy script for extensions by @@samansmink in #10063 * Add list_reduce lambda function by @@maiadegraaf in #9909 * Handle s3_endpoint that includes path by @@tom-s-powell in #9918 * Update year in license file to 2024 by @@szarnyasg in #10103 * Not including stdlib for just size_t, stddef is smaller header by @@hannes in #10104 * Bump iceberg extension version by @@samansmink in #10102 * Internal #783: ICU DatePart Serializers by @@hawkfish in #10055 * CI: Add backlink to original issue when its status changes by @@szarnyasg in #10111 * Streamline handling issues that come with PRs fixing them by @@szarnyasg in #10116 * Fix ParquetScanMaxThreads. by @@yiyuanliu in #10113 * test(python): fix blind catches by @@Mause in #10118 * CI: Create separate job for handling 'PR submitted' label by @@szarnyasg in #10119 * Lazy WAL creation by @@taniabogatsch in #10086 * [BREAKING] Modify implicit casting rules to differentiate between string literals and VARCHAR columns, and disable implicit casting to VARCHAR for many types by @@Mytherin in #10115 * For parsed expressions and table references - serialize query location, and obtain query location for more properties in transformer by @@Mytherin in # 10130 * Checksum WAL entries by @@Mytherin in #10126 * Display unnamed structs as tuples instead of as structs with empty keys by @@Mytherin in #10134 * Fix #10008 - disallow parameters in DEFAULT clause, and remove unsupported SQLite code for handling parameters in shell by @@Mytherin in #10135 * Issue #10140: NULL Constant Lists by @@hawkfish in #10142 * feat(py): support python 3.12 by @@Mause in #10144 * fix create_function python stub by @@yiyuanliu in #10132 * fix(py): don't use jemalloc on windows by @@Mause in #10145 * Build manylinux_2_28 arm python wheels by @@samansmink in #10137 * Slightly improve performance of the first aggregate function by @@lnkuiper in #9957 * Add csv & parquet write functions and toPandas to experimental PySpark API by @@TomBurdge in #9672 * Add handling for duckdb-wasm extensions in extension-upload-from-nightly by @@carlopi in #9883 * Add some pg session function to pg_catalog by @@goldmedal in #10156 * Fix CSE elimination for window functions with bind data. Fixes #10124 by @@How-u-doing in #10152 * feat(py): Py3.12 support for Windows by @@Mause in #10159 * Py3.12 Windows nightly fixes by @@Mause in #10160 * Serialization for scalar functions nextval and currval by @@ywelsch in # 10146 * Pushdown filters into semi and anti joins by @@Tmonster in #10110 * Issue #9950: Ordered Aggregate Performance by @@hawkfish in #10045 * uhugeint_t/hugeint_t operator changes by @@nickgerrets in #10117 * add regexp_escape function by @@chrisiou in #10044 * Infrastructure: truncate not always available in CI, use dd by @@carlopi in #10162 * JSON S3 optimization by @@samansmink in #10151 * Remove chunk collection from reservoir sampler by @@Tmonster in #10038 * Fix #10074 - for materialized CTEs the final result names are not influenced by the aliases. Only the names of the CTE itself are influenced by the aliases. by @@Mytherin in #10163 * [Python] Fix issue in DataFrame construction where non-deduplicated names were being used mistakenly by @@Tishj in #10165 * Fix #10141 - Correctly handle recursive and nested types that refer to other types in CREATE TYPE by @@Mytherin in #10164 * Suspend duckdb shell on Ctrl+Z by @@gsauthof in #10172 * Fix #9456 and improve Deliminator by @@lnkuiper in #9993 * Temporary Memory Manager by @@lnkuiper in #10147 * Issue #10138: Finite Temporal Helpers by @@hawkfish in #10157 * Internal #425: TIMETZ Functions by @@hawkfish in #10107 * Fix #10057: report correct error message when binding an aliased column fails by @@Mytherin in #10176 * Allow unquoted keywords to be used in DETACH by @@Mytherin in #10175 * Implement FILE_SIZE_BYTES by @@lnkuiper in #9920 * [Parquet] Fix 10148, allow reading large byte arrays into small decimals for stats by @@hannes in #10183 * Throw binder error when returning list has no columns by @@Tmonster in # 10184 * Fix missing move by @@samansmink in #10186 * [BREAKING] Implicit cast rules for integer literals, and parameterized ANY for binding by @@Mytherin in #10194 * Handle 0-list maps (erroring out) and add test-case by @@carlopi in #10187 * Internal #1001: SEM Test Determinism by @@hawkfish in #10197 * More work towards Custom Indexes by @@Maxxen in #10032 * Collations are not yet properly serialized, remove enable_verification by @@carlopi in #10206 * fix: reject creation of persistent secrets when allow_persistent_secrets= false for storage backends other than local_file by @@stephaniewang526 in # 10199 * prevent physical nested loop join with multiple conditions on semi an?? by @@Tmonster in #10190 * chore: add const to BaseSecret and KeyValueSecret copy ctrs, fix compilation error by @@stephaniewang526 in #10182 * Allow DESCRIBE/SHOW/SUMMARIZE to be used as a subquery by @@Mytherin in # 10210 * Internal #1042: 2023d Time Zones by @@hawkfish in #10215 * fix: fix a bug where free_space was incorrectly calculated when flushing partial blocks by @@gitccl in #10220 * Support more PG information_schema views by @@goldmedal in #10222 * Fix Issue #10122: wrong result in IEJoin by @@xuke-hat in #10123 * Internal #861: Aggregation Absorb API by @@hawkfish in #9971 * Better Cardinality estimates for right and left semi/anti joins by @@Tmonster in #9976 * Open JSON files lock-free if there are many by @@lnkuiper in #10188 * Bitpacking compression for the UHUGEINT type by @@nickgerrets in #10195 * fix: drop secret if exists on non-existing secret should not cause ex by @@stephaniewang526 in #10217 * Replace 'embedded' with 'in-process' in the Python package description by @@szarnyasg in #10232 * In .mode json in the shell use standard float output method by @@Mytherin in #10229 * Remove duplicate columns in PhysicalHashJoin by @@lnkuiper in #10203 * Out-of-tree extensions for R Windows by @@samansmink in #10204 * [Python] Output unnamed structs as tuple in fetchone/many/all methods by @@Tishj in #10174 * Internal #1022: Window TIME RANGE by @@hawkfish in #10181 * fix(capi): add basic TIME_TZ support by @@Mause in #10207 * Lambda scoping by @@taniabogatsch in #10150 * Add batch functionality by @@osidekyle in #10011 * Make FILE_SIZE_BYTES test more lenient by @@lnkuiper in #10234 * Allow SQLNULL to be bound in VALUES list by @@Mytherin in #10238 * Sequence clean-up - move all sequence access through a SequenceData entry that is concurrency safe by @@Mytherin in #10236 * Remove duplicate join conditions by @@lnkuiper in #10235 * UNPIVOT - maintain original types and throw a better exception if type matching is not possible by @@Mytherin in #10240 * Correctly account for ORDER BY in ColumnLifetimeAnalyzer optimization pass by @@Mytherin in #10242 * CSV Parser 2.0 by @@pdet in #10209 * COPY - allow copy_file_name to be formatted as identifier by @@Mytherin in # 10246 * remove C-style cast in compression module by @@gitccl in #10256 * Allow dsdgen to be interrupted by user by @@elefeint in #10262 * Upgrade the schema of pg_proc to PostgreSQL 16 by @@goldmedal in #10248 * chore: improvements to duckdb_api / user_agent by @@elefeint in #10226 * Issue #10249: Window Clause Casing by @@hawkfish in #10259 * Fix issue 10254 by @@lnkuiper in #10265 * Store unqualified macro parameters, qualify before binding by @@lnkuiper in #10266 * Fix 9384 by @@lnkuiper in #10267 * Fixes #10212: Correctly account for non-flat vectors in ListColumnData::Skip by @@Mytherin in #10268 * Enable intercepting file copy extension after binding by @@Maxxen in #10255 * [Dev][Util] Have unittest respect the --start-offset parameter when used alongside -l by @@Tishj in #10277 * Issue #10272: DATE + INTERVAL by @@hawkfish in #10274 * Fix #10279 - correctly use unsigned integers in delta decoding to wrap overflows as specified by the spec by @@Mytherin in #10288 * Add parse_path(), parse_dirname() and parse_filename() functions by @@chrisiou in #10208 * [Compression] ALP Compression (float/double) by @@lkuffo in #9635 * Add function for non-bias-corrected kurtosis by @@david-cortes in #9545 * fix the problem of fetching wrong data when using bitpacking compression by @@gitccl in #10291 * [Python] Compatibility with pandas==2.2.0 by @@Tishj in #10290 * Avoid re-rendering the progress bar if the percentage has not changed by @@Mytherin in #10284 * Adapt further for z/OS by @@v1gnesh in #10297 * CSV Parser Optimizations: Pre-computed skip list. Simpler state machine transition. Flip state machine states and transitions. by @@pdet in #10258 * Explicitly cast FTS column to VARCHAR by @@lnkuiper in #10299 * Add table macro_definition to duckdb_functions() by @@lnkuiper in #10301 * [CSV Parser] Fix regression with dialect detection on quoted values. by @@pdet in #10307 * Improve bitpacking skip performance + better testing of FetchRow by @@samansmink in #10295 * Fix #10308 - allow describe/summarize as prepared statement, and fix issue with describe/summarize with ctes by @@Mytherin in #10311 * CSV Reader making null padding Parallel by default by @@pdet in #10306 * Remove spinlock for closing JSON files by @@lnkuiper in #10300 * Fix NumericLimits::Minimum() * 0 case by @@xuke-hat in #10319 * C API updates by @@taniabogatsch in #10317 * Add create_sort_key function by @@Mytherin in #10321 * SQLLogicTestRunner - Make mode command work in loops, and add mode no_output by @@Mytherin in #10328 * Escape all ASCII control characters in duckbox rendering by @@Mytherin in # 10327 * Fix Failing Nightly Swift test by @@Tmonster in #10320 * Basic struct filter pushdown by @@Maxxen in #10314 * Improve the check for glob in sniff_csv function. by @@gabihodoroaga in # 10243 * Add an icon to the duckdb.exe shell executable by @@renevdzee in #9656 * [Arrow] Support scanning REE (Run End Encoded) Arrow arrays by @@Tishj in # 9836 * [Compression] Fix Handling of -0.0 in ALP by @@lkuffo in #10335 * Split jdbc tests by @@Mause in #10338 * Set interrupted flag after pushing an error to prevent race condition where the InterruptException could end up being the top-level error by @@Mytherin in #10337 * [Python] Fix bug in conversion of INTERVAL to datetime.timedelta by @@Tishj in #10339 * Update Postgres and SQLite extensions by @@Mytherin in #10310 * Limit the number of threads used to scan distinct aggregates by @@lnkuiper in #10318 * Fail Regression test when difference in cardinalities is detected by @@Tmonster in #10340 * pin ccache action version by @@samansmink in #10344 * test_all_types.py: Formatting according to black 24 by @@carlopi in #10353 * Allow NULL bytes in ART indexes by @@Mytherin in #10325 * Ensure version numbers passed to Windows .rc file are numbers by @@Mytherin in #10358 * Feature: Digit separators in numeric literals by @@Maxxen in #10343 * Autocomplete small improvements by @@carlopi in #10370 * Fix: correctly set list size of dictionary vector by @@gitccl in #10369 * Check black version in script/format.py by @@carlopi in #10354 * Change extension_directory default by @@carlopi in #10359 * feat(jdbc): setBytes by @@Mause in #10365 * [Block Size] CI test for 16KB block size and related code changes by @@taniabogatsch in #9967 * CI: Add workflow to check new issues for code formatting by @@szarnyasg in # 10149 * Add additional STRUCT expansion tests by @@taniabogatsch in #10373 * Fix MATERIALIZED CTE issue #10260 by @@kryonix in #10386 * Add table_sample to TableStatistics (currently saved as nullptr) by @@Tmonster in #10378 * [Python][Dev] Use duckdb_cursor to avoid unintentionally sharing catalogs and registered objects by @@Tishj in #10388 * Extension metadata: detecting the platform by @@carlopi in #10329 * Add missing checkpoints in test by @@Maxxen in #10389 * Remove Python function signature (in test) by @@carlopi in #10399 * Avoid requiring expected error message on original sqlite tests by @@carlopi in #10392 * Case insensitive extensions install & load by @@carlopi in #10380 * Platform detection: Fix thread sanitizer job passing absolute path by @@carlopi in #10398 * [CSV Reader] Implicit Casting and Projection Pushdown by @@pdet in #10390 * CI: Fix workflow to check new issues for code formatting by using heredoc by @@szarnyasg in #10403 * Rework Exception Internals by @@Mytherin in #10410 * avoid copying in the for loop when there is a const by @@zhouzilong2020 in # 10418 * Fix drop secret bug by @@samansmink in #10185 * Issue template: Ask about testing with the nightly build (instead of 'main') by @@szarnyasg in #10405 * [CSV Reader] Use array instead of unordered_map in projection pushdown by @@pdet in #10411 * [C-API] Add duckdb_appender_column_type by @@maiadegraaf in #10401 * Fix results for anti joins on empty tables by @@Tmonster in #10413 * Don't write Chimp/Patas files anymore & deprecate patas and chimp by @@Mytherin in #10423 * Small secret manager refactor by @@samansmink in #10421 * [Python][Dev] Skip pyarrow test_struct_filter_pushdown on python3.8 by @@Tishj in #10419 * Add read_text and read_blob table functions by @@Maxxen in #10376 * Rework FunctionSideEffects to FunctionStability - allow NOW() to be pushed down by @@Mytherin in #10426 * Remove dev logging on local_extension_repo creation by @@carlopi in #10424 * GH Workflows: Create CI job for Coverity scan by @@moshekaplan in #10433 * Issue #10224: FIRST ORDER BY by @@hawkfish in #10347 * Coverity Scan: Project name is DuckDB by @@Mytherin in #10447 * Multiline mode: add continuation prompt rendering by @@Mytherin in #10425 * Python: fix exception hierarchy, and also catch std::exception by @@Mytherin in #10420 * Fix bug with full file download by @@samansmink in #10429 * Add cloudflare invalidation to nightly deploy script by @@samansmink in # 10430 * fix(py): fix building python binding from cmake by @@Mause in #10445 * [CSV] Bug Fix related to quoted values starting with empty values by @@pdet in #10438 * Add COMMENT ON statement by @@samansmink in #10372 * CLI: Highlighting for continuation tokens by @@Mytherin in #10453 * Issue #10224: ORDERED FIRST Rewrite by @@hawkfish in #10457 * add support for date, time, timestamp types to the Julia appender api by @@rdavis120 in #10449 * Bugfix/#10441 correctly validate second colon in jdbc url parameter in Java client. by @@peteraisher in #10442 * [Execution] Parallel StreamQueryResult by @@Tishj in #10245 * [Julia][Dev] Improve README by @@Tishj in #10464 * Linenoise Code Cleanup by @@Mytherin in #10461 * [Bug Fix] [CSV Sniffing] Removing double quotes in header by @@pdet in # 10462 * [Add Test] [CSV Parser] Type Detection on columns with null values by @@pdet in #10465 * Unconnected fixes by @@carlopi in #10467 * CLI: Only show continuation bytes while editing the query by @@Mytherin in # 10470 * [CSV Reader] [Add Test] Quoted Values impacting the column sniffing. by @@pdet in #10472 * GzipFS - use unique_ptr instead of new/delete by @@Mytherin in #10474 * Group together index instantiation parameters into a struct, pass options map as well. by @@Maxxen in #10471 * Patch to build on Windows ARM 64 by @@hannes in #10479 * [Bug Fix] [CSV Reader] Fix to using null_padding in conjunction to one of the multifile reader options by @@pdet in #10473 * More linenoise/CLI improvements by @@Mytherin in #10480 * Add documentation example for Julia appender api. by @@rdavis120 in #10475 * Linenoise: add support for many more alt command sequences by @@Mytherin in #10493 * Use CMAKE_CURRENT_BINARY_DIR in CMakeLists by @@Flogex in #10484 * Avoid setting DUCKDB_NORMALIZED_VERSION to an empty string by @@SChakravorti21 in #10492 * Tag memory that is allocated through the buffer manager, and add duckdb_memory() function by @@Mytherin in #10496 * Linenoise: Disable automatic auto-complete rendering by default for now by @@Mytherin in #10498 * Fix edge case in RANGE for dates/timestamps when start=end by @@Mytherin in #10499 * Fix for auto-complete on empty words so behavior is consistent with previous DuckDB by @@Mytherin in #10502 * Nested Array Validity Fixes + RowOperations by @@Maxxen in #10483 * Keep file extension for temporary files by @@Maxxen in #10459 * bundle-library: Add optional Makefile target by @@carlopi in #10507 * Fix #10363 - prefer selecting entire rows over SQL value functions by @@Mytherin in #10505 * [CSV Reader] Avoid unnecessary writes to temporary file when reading in limited memory cases by @@pdet in #10509 * [Dev] Fix ASAN thread limit exceeded issue in CI by @@Tishj in #10510 * [Dev] Cancel tasks in EndQueryInternal by @@Tishj in #10512 * Export stack trace as part of extra exception info if DUCKDB_DEBUG_STACKTRACE is defined (DEBUG_STACKTRACE cmake variable) by @@Mytherin in #10514 * feat: make SecretEntry copyable so we don't lose the underlying BaseSecret obj for non-CatalogSet secrets by @@stephaniewang526 in #10518 * fix(py): json type in description field by @@Mause in #10521 * Decorrelation and parallelization of recursive and materialized CTEs by @@kryonix in #10357 * Keep track of view names and aliases separately so we can distinguish between explicitly provided aliases by the user and names returned by the view, and block usage of view if names changed by @@Mytherin in #10500 * Tweak TemporaryMemoryManager by @@lnkuiper in #10503 * On *nix, return errno in IOException extra_info by @@philippmd in #10529 * Use null cast instead of reinterpret for NULL -> JSON by @@lnkuiper in # 10526 * [Python] Fix various small issues by @@Tishj in #10533 * [CSV Reader] NullPadding Tests by @@pdet in #10530 * Linenoise: make ENTER behave more similarly to other CLI clients - Ctrl+X can now be used to enter newlines at the cursor position by @@Mytherin in # 10527 * [Python][Dev] Add PYTHON_EDITABLE_BUILD to Makefile by @@Tishj in #10488 * Reset git status before applying patches in out-of-tree build by @@Mytherin in #10534 * Fix #10486 - allow out-of-order struct casting by @@Mytherin in #10537 * Make ClientContext available during attach by @@ywelsch in #10531 * feat(jdbc): setDate by @@Mause in #10408 * Fix unused variable warning/error by @@carlopi in #10542 * [Dev]: ICU 2024a TimeZones by @@hawkfish in #10544 * Path utility fixes by @@Maxxen in #10538 * Support struct_extract and unnest for unnamed structs by @@Mytherin in # 10541 * [Dev] Fix isses in parallel Checkpoint by @@Tishj in #10525 * More TemporaryMemoryManager tweaks by @@lnkuiper in #10549 * Add order to tests by @@Flogex in #10536 * Require at least 1 argument when calling ListZip during bind by @@Tmonster in #10485 * Benchmark runner: check for errors also if no result is specified by @@Mytherin in #10554 * [CSV Reader] [Bug Fix] Nightly CI Segfaults by @@pdet in #10551 * Fix for test/sql/copy/file_size_bytes_large.test_slow test by @@pdet in # 10552 * Simplify binary-search in FindRunIndex by @@felipecrv in #10487 * Remove query profiler history and add hooks for certain events in the ClientContext by @@Mytherin in #10504 * [CSV Reader] Reset state on quoted/escaped when applying projection pushdown by @@pdet in #10557 * CommitState::WriteCatalogEntry refactor by @@carlopi in #10555 * [CSV Reader] Fix 9952 and race condition on error handler by @@pdet in # 10558 * Fix a few tests failing non-deterministically with ALTERNATIVE_VERIFY=1 by @@carlopi in #10560 * unittester: if test-dir is provided avoid deleting it by @@carlopi in #10564 * Allow DuckDB execution without implicit main thread by @@ywelsch in #10548 * Fix #10528 - disallow parsing exponents for integers in strict parsing mode by @@Mytherin in #10559 * [Python] Fix issues related to handling of Python exceptions by @@Tishj in # 10563 * [Python] Fix crash caused by fetch_record_batch by @@Tishj in #10565 * [Dev] Skip some tests on ALTERNATIVE_VERIFY by @@Tishj in #10567 * [Python][Dev] Test does not throw HTTPException by @@Tishj in #10568 * [Export] Fix export of user-defined types by @@Tishj in #10569 * Fix heap buffer overflow in VARCHAR -> TIME trycast by @@Tishj in #10571 * [Dev][CI] Add --no-exit to run_tests_one_by_one by @@carlopi in #10572 * Handle also branch with no threads enabled by @@carlopi in #10573 * Deduplicate code dealing with deduplication of column names by @@Tishj in # 10532 * Bump spatial by @@Maxxen in #10580 * extension_distribution.yml: Pass DUCKDB_PLATFORM by @@carlopi in #10582 * chore(jdbc): correct datetime delta by @@Mause in #10584 * Fix Nested Array TupleData Serialization by @@Maxxen in #10513 * Do not replace filters that evaluate to always true by @@Tmonster in #10553 * [Dev] Fix triggered assertion in SortedAggregateState::FlushChunks caused by a small STANDARD_VECTOR_SIZE by @@Tishj in #10202 * [Tester] Add --require by @@Tishj in #10579 * Add Support Options page by @@szarnyasg in #10598 * Use all threads to read multiple parquet files. by @@Tmonster in #10590 * [CSV-Reader] Fix on finalize for projection pushdown + nullpadding by @@pdet in #10596 * [Julia] Remove DataFrame from Project.toml by @@Tishj in #10605 * [CSV Reader] [Bug Fix] Make CSV Results hold the buffers they depend on by @@pdet in #10589 * minor secret manager fix by @@samansmink in #10600 * Point error message to stable link by @@carlopi in #10592 * Fix issue in aggregate HT where a task could be blocked and never be unblocked if the aggregation was interrupted early due to e.g. a limit by @@Mytherin in #10604 * Memory tracking - explicitly zero-initialize memory tracking arrays by @@Mytherin in #10606 * [Dev] Bump extensions & apply patches by @@carlopi in #10603 * Secret folder by @@carlopi in #10607 * JDBC: Skip combine test run because #10338 by @@hannes in #10616 * Fix for SQL value functions when there is an alias specified by @@Mytherin in #10611 * delay secret storage initialization by @@samansmink in #10612 @ text @d1 1 a1 1 # $NetBSD: Makefile,v 1.2 2023/12/01 15:29:29 ryoon Exp $ d20 16 a35 16 CMAKE_ARGS+= -DBUILD_CORE_FUNCTIONS_EXTENSION=ON CMAKE_ARGS+= -DBUILD_ICU_EXTENSION=ON CMAKE_ARGS+= -DBUILD_PARQUET_EXTENSION=ON CMAKE_ARGS+= -DBUILD_TPCH_EXTENSION=ON CMAKE_ARGS+= -DBUILD_TPCDS_EXTENSION=ON CMAKE_ARGS+= -DBUILD_FTS_EXTENSION=ON CMAKE_ARGS+= -DBUILD_HTTPFS_EXTENSION=ON CMAKE_ARGS+= -DBUILD_VISUALIZER_EXTENSION=ON CMAKE_ARGS+= -DBUILD_JSON_EXTENSION=ON CMAKE_ARGS+= -DBUILD_JEMALLOC_EXTENSION=ON CMAKE_ARGS+= -DBUILD_EXCEL_EXTENSION=ON CMAKE_ARGS+= -DBUILD_INET_EXTENSION=ON CMAKE_ARGS+= -DBUILD_BENCHMARKS=ON CMAKE_ARGS+= -DBUILD_SQLSMITH_EXTENSION=ON CMAKE_ARGS+= -DBUILD_TPCE=ON CMAKE_ARGS+= -DBUILD_AUTOCOMPLETE_EXTENSION=ON @ 1.2 log @duckdb: Fix C/C++ standard handling Pointed out by gdt@@. Thank you. @ text @d1 1 a1 1 # $NetBSD: Makefile,v 1.1 2023/12/01 10:45:58 ryoon Exp $ d3 1 a3 1 DISTNAME= duckdb-0.9.2 @ 1.1 log @databases/duckdb: import duckdb-0.9.2 DuckDB is a high-performance analytical database system. It is designed to be fast, reliable and easy to use. DuckDB provides a rich SQL dialect, with support far beyond basic SQL. DuckDB supports arbitrary and nested correlated subqueries, window functions, collations, complex types (arrays, structs), and more. @ text @d1 1 a1 1 # $NetBSD$ d14 1 a14 1 USE_LANGUAGES= gnu99 gnu++11 d18 1 a18 1 BUILDLINK_TRANSFORM+= opt:-std=c++11:-std=gnu++11 @