Changelog
Source:NEWS.md
data.sketches 0.1.2
Security
The bundled Apache DataSketches C++ library is updated from 5.2.0 to 5.3.1, which fixes out-of-bounds reads and writes when deserializing corrupt KLL, REQ, HLL, CPC, Count-Min, Theta and VarOpt payloads (CVE-2026-103501, CVE-2026-103513, CVE-2026-103634, CVE-2026-103635, CVE-2026-103636, CVE-2026-123274, CVE-2026-123338). Every
bytes =constructor was affected. If you rebuild sketches from bytes you did not produce, upgrade.Many deserialization bugs that remain in 5.3.1 are patched locally, found by fuzzing every
bytes =constructor under AddressSanitizer and UndefinedBehaviorSanitizer (#33). A corrupt payload is now rejected withdatasketches_invalid_bytesinstead of reaching undefined behavior, an out-of-bounds read or write, a double free, a crash on first use (a Bloom filter of zero capacity divided by zero on$update()), or an allocation of gigabytes from a few bytes. The families affected arehll(),bloom_filter(),ebpps(),frequent_items(),req(),count_min(),tdigest_double(),theta(),array_of_doubles(),kll_doubles(),kll_floats()andvaropt(). Payloads written by valid sketches, including those from data.sketches 0.1.1, still load.A small corrupt
req()payload could request an 8 GiB allocation; 5.3.1 rejects it (#33).
Bug fixes
A
frequent_items()sketch that ran out of memory while growing, or a KLL sketch that ran out of memory in$merge(), no longer crashes R later, when it is garbage-collected or R exits (“double free or corruption”). The library left the sketch’s memory half-replaced after the failed allocation; both are patched locally (#35).tdigest_double()no longer returnsInforNaNquantiles for data near the largest double, and such a sketch can again be read back after$serialize(). The library multiplied means by weights before dividing, which overflowed; it is patched locally to divide first.bloom_filter()validates sizes against the library’s own maximum, 17,179,868,864 bits (2 GiB):num_bitsabove it raisesdatasketches_invalid_num_bits, andmax_itemswhose filter atfppwould exceed it raisesdatasketches_invalid_max_items, naming the limit and what to change. Such sizes used to pass R’s check (up to 2^53) and fail in the library with an unclassed error, fornum_bitswith the wrong limit (“less than 2^63”) in its message.bloom_filter_suggest_num_hashes()gets the same check;bloom_filter_suggest_num_filter_bits()still reports what a size needs, even past the limit (#32).bloom_filter_suggest_num_hashes()suggests at most 65535 hash functions, the most a filter stores. With more than about 94,500 bits per item the library’s count overflowed its 16-bit result, which is undefined behaviour; it is patched locally to clamp.Every native constructor’s error is now classed: one the library raises that R’s checks did not anticipate arrives as
datasketches_invalid_args, with the library’s message as its parent, sotryCatch(datasketches_error = )catches every failure as 0.1.1 promised (#32).req(k =)now honours widths of 256 and above. The library storedkin 8 bits, sok = 500built a sketch of width 244 andk = 256ork = 1024one of width 4, with correspondingly worse accuracy.
Breaking changes
varopt()and$update()on a VarOpt sketch reject aweightof zero, withdatasketches_invalid_weight. The library now requires positive weights, as datasketches-java and the VarOpt paper do; zero-weight items never contributed to an estimate.Results from the updated library can differ slightly from 0.1.1: t-Digest ranks and quantiles in the tails, and HLL union estimates, which no longer depend on merge order. Serialized HLL sketches and single-item Theta sketches now match datasketches-java byte for byte; payloads written by 0.1.1 still load.
data.sketches 0.1.1
CRAN release: 2026-09-23
Bug fixes
Interrupting a long
$update()or query no longer leaks its input vector. The native loops probed for interrupts withR_CheckUserInterrupt(), which longjmps past C++ destructors and so skipped the release of cpp11’s preserve token on the input; an interrupted 40M-element update pinned ~320 MB in R’s precious list for the rest of the session. All 25 probe sites now usecpp11::check_user_interrupt(), which unwinds throughR_UnwindProtect.theta()andarray_of_doubles()sketches rebuilt frombytes =no longer lose their width. A compact payload does not carry the builderlg_k, and the previous fallback to the default width silently cost about an order of magnitude of accuracy on any later$merge()or union — unioning two round-trippedlg_k = 20sketches retained 4,096 entries instead of 1,048,576. The width is now recovered from the retained-entry count.Native update and query loops no longer pump the R event loop on the first element of every call.
New features
-
$merge()ontheta_sketchandarray_of_doubles_sketchgains anlg_kargument, matchingtheta_union()andarray_of_doubles_union(). Previously there was no way to control the width of a merge.
Error handling
Errors originating in the C++ bridge now carry the same classed conditions as the R-level validators, so
tryCatch(datasketches_error = )catches every failure the package raises. New classes:datasketches_invalid_bytes(corrupt or truncatedbytes =payload),datasketches_empty_sketch($quantile(),$rank(),$cdf(),$pmf(),$min(),$max()on an empty sketch), anddatasketches_dead_pointer.Printing or summarising a sketch restored with
readRDS()orload()now raises an actionable error pointing at$serialize()/bytes =, rather than reporting that the object is not a valid sketch. External pointers do not survive R’s serialization; sketches must be persisted with$serialize().A
bytes =payload shorter than 8 bytes passed tohll()now raisesdatasketches_invalid_bytesrather thandatasketches_invalid_args, so a malformed payload has one class across all sketch families. Both classes inheritdatasketches_error.
data.sketches 0.1.0
CRAN release: 2026-07-09
- Initial release. Provides an R interface to the Apache DataSketches C++ library:
- Quantile sketches:
kll_doubles(),kll_floats(),req(), andtdigest_double(). - Cardinality sketches:
hll(),cpc(), andtheta()(with set operations). - Frequency sketches:
frequent_items()andcount_min(). - Tuple sketches:
array_of_doubles()(with set operations). - Sampling sketches:
varopt()andebpps(). - Filters:
bloom_filter().
$update(),$merge(),$summary(),$inspect(), and native$serialize()/bytes =round-tripping, plusformat(),print(),as.character(), andsummary()methods. - Quantile sketches: