Changelog
Source:NEWS.md
decimal 0.1.1
decimalvectors now convert to and from ‘Arrow’ decimal arrays directly.as_decimal()gained methods forArrayandChunkedArray. An Arrow decimal column of any width converts exactly, taking its scale from the Arrow type: Arrow’s own decimal-to-string cast produces this package’s canonical storage form once the exponent letter is folded to lowercase, so the strings need no re-parsing, and a property test guards that invariant. For 1,000,000decimal128()values that takes 0.27s against 0.78s for casting to string and callingdecimal(). Arrow integer columns convert exactly at every width, includingint64anduint64values a double cannot hold. Other Arrow types convert through the ordinaryas_decimal()rules for the equivalent R vector.A
decimalcolumn now becomes an Arrow decimal field inarrow::arrow_table(),arrow::write_parquet()andarrow::write_dataset(). The field is an Arrow extension type whose storage is a realdecimal128()ordecimal256(), so Spark, DuckDB, pandas and other readers see a plain decimal column, while in R it returns as adecimalvector on every read path:as.data.frame(),arrow::read_parquet()anddplyr::collect()included. Newarrow_decimal_type()pins a precision and scale for a column that will be appended to. Infinities and NaNs, which no Arrow decimal can represent, now raise an error on conversion instead of round-tripping through the ‘vctrs’ extension type; keep such a column as a string if you need them.Arrow’s compute engine does not operate on extension columns. For arrow-side arithmetic or filtering on a decimal column, write a plain field by passing a plain Arrow decimal type to
arrow::as_arrow_array()or by settingoptions(decimal.arrow_extension = FALSE). A plain field comes back from arrow’s own conversion as a double wearing thedecimalclass, because arrow reapplies the column’s recorded R attributes;format()andas.character()now refuse such an object rather than print rounded values. Newarrow_as_data_frame()converts an ArrowTableorRecordBatchto a data frame, reading plain decimal fields, including those in files written by other systems, asdecimalvectors, and leaving every other column to arrow.Fixed memory leaks on the error paths of the native kernels. An R error raised while an
mpd_twas allocated abandoned it, because R’s error handling unwinds past the code that would have freed it. Parsing an invalid decimal string leaked every handle the operation held, and an allocation failure part-way through setting up an operation leaked the handles already allocated. An R allocation failure while storing a result string leaked the formatted text too; that text is now released underR_UnwindProtect().summary()now works ondecimalvectors. It reports the same six statistics as [summary.default()] plus anNA'scount, computed in decimal arithmetic and returned as adecimalvector, so figures that a double would round are preserved. Quartiles use the type 7 definition, matchingstats::quantile().Fixed
is.na(),is.nan(),is.infinite(),is_qnan()andis_snan()returning malformed logical vectors. They forwarded mpdecimal’s flag bits – 4 forNaN, 8 forsNaN, 2 forInfinity– instead ofTRUE. The results printed correctly and compared equal with==, butwhich()found no matching elements,sum()over-counted, andidentical()was false against the obvious expectation.DESCRIPTIONnow declaresURLandBugReports, so the CRAN page links to the source repository and the issue tracker.Building a
decimalvector from character is substantially faster. The fractional-digit scan thatdecimal()performs on every element moved from R to C, which for 100,000 values takes construction from 1.13s to 0.05s whenscaleis supplied (about 22x) and from 2.28s to 0.05s when it is inferred (about 45x). Inferring the scale now costs the same as supplying it. This matters most when loading decimal data from ‘Arrow’ or a database, where the values arrive as a character vector in one call.New vignette
vignette("arrow-decimal-types"), covering lossless conversion betweendecimalvectors and Arrow’sdecimal128()/decimal256()types in both directions, including chunked arrays, Parquet columns, and the cases Arrow decimals cannot represent.format()on adecimalvector now accepts the extra arguments that table printers pass to every column, such asdigits,na.encodeandjustifyfromformat.data.frame(),timezonefrom ‘data.table’ andtrimfromknitr::kable(), so decimal columns print inside a basedata.frame, adata.tableand akable()table. The arguments are ignored rather than honored, so an exact value is never silently rounded for display. Arguments that would change how a number is written, such asnsmall,scientificandbig.mark, are an error.Fixed
rbind()on data frames withdecimalcolumns, which always failed with “Can’t assign to elements past the end”. Assigning past the end of adecimalvector now grows it with missing values, as it does a base vector, and binding columns of different scales takes the finer one.x[] <- valuenow replaces every element instead of failing.match(),%in%and basemerge()now comparedecimalvalues rather than their stored text, so2.5matches2.50, as==already said. Whole numbers still match integers and strings:decimal("20") %in% 20LstaysTRUE.New section in
vignette("decimal-values")on decimal columns in a ‘data.table’: what works, and workarounds for the operations data.table runs on the stored text instead of the values, such as sorting and groupedmin()andmax().