- Final release for StoX 4.1.0.
- Fixed bug in DefineTranslation, where the ConditionalVariableNames was showing as a single string and not a vector.
- Fixed bug in asIntegerAfterRound() used when setting class of ICES data to avoid floating point to integer errors. The bug appeared when the input was character (but convertible to numeric).
- Fixed warnings in translateOneTable() so that a warning is given if the variable to translate is not present in any table, and if any conditional variables are not present in a table to be translated.
- Applied ‘release’ and ‘oldrel’ in the check-full.yaml, securing that binaries are built for the current and previous R minor versjon.
- Fixed bug in as.numeric_IfPossible() used by setorderv_numeric() and orderRowsByKeys() where individual elements could be set to NA in a vector unless all of the values were NA after conversion to numeric. In the new version all of the values must be convertible to numeric for a numeric vector to be returned. In addition setorderv_numeric() has gained the parameter split, which is used in RstoxBase::formatOutput() as split = c(“-”, “/”) to split both by the within StoX key separator and the between StoX kye separator used in IDs such as Sample and Individual. This bugfix may result in different sorting of StoxBiotic, particularly for NMDBiotic data with herring coded as catchcategory 161722.G03, 161722.G05 or 161722.G07.
- Changed the drop down list of ConditionalVariableNames in Translate functions to only include the variables in the table of the VariableName (and also excluding the VariableName). Previously all variables of the entire data were listed, which was confusing since only those present in the relevant table could be used.
- Changed the behavior of Translate functions when a variable that is not present in the table is used as a conditional variable. Before this conditional variable was effectively ignored, but in the new version the behavior is to give a warning and not perform any translation.
- Added a warning if no values are translated in Translate functions.
- Added the new GeneticPopulationCode to ICESBiotic().
- Temporarily hiding Prey tables in StoxBiotic().
- Fixed a bug where certain values of BiologyLengthCode were shifted one integer value down in ICESBiotic(). The bug is related to floating point precision which causes some values to be slightly lower than the corresponding integer after calculations. In R one example is format(29 / 100 * 100, digits = 20) = “28.999999999999996447”, which results in 28 when converted to integer. The following values are affected:
- 29, 57, 58, 113, 114, 115, 116 when BiologyLengthCode is “cm” (lengthresolution “3”)
- 1001, 1003, 1005, 1007, 1009, 1011, 1013, 1015, 1017, 1019, 1021 and 1023 when BiologyLengthCode is “mm” (lengthresolution “1”)
- 1005 and 1015 (a subset of the values for “mm”) when BiologyLengthCode is “halfcm” (lengthresolution “2”)
- Added SpeedGround as vesselspeed from NMDBiotic in ICESBiotic().
- Fixed bug in RedefineStoxBiotic(), where duplicated keys in the input BioticData were warned but not removed.
- Added documentation of PreySpeciesCategory and PreySample in StoxBiotic.
- Fixed bug where PreyCatchfractionWeight = 0 was set to NA if PreyCatchFractionWeightResolution was missing. Now returning 0.
- Fixed inaccuracies in the documentation of the StoxBiotic format.
- Added warning when there are missing values in keys in StoxBiotic.
- Changed mapplyOnCores() to using sockets both for Windows and macOS, which solved the problem that the memory of the parent R session was copied to all cores, potentially causing memory issues
- Added the argument ignore.condition to applyFunctionArgumentHierarchy() which is now used by RstoxFramework to get functions inputs that are hidden by UseProcessData. See news of RstoxFramework.
- Added tables PreySpeciesCategory and PreySample in StoxBiotic, and prepared for adding PreyIndividual.
- Fixed bug where reports could be run even though the Baseline model had been rerun.”
- Final release for StoX 4.0.0.
- Made comparison to ICES vocabularies more robust, and re-introduced the test for ICESExport.
- Renamed backwardCompatibility to backwardCompatibility_RstoxData to avoid the same name to be used across Rstox packages, and restructured functions in that list to use inputs projectDescription, modelName, processIndex and not projectDescriptionOne, allowing for using information from the entire project.
- Temporarily disabled test for ICESExport.
- Fixed bug in rbindlist_StoxFormat() where elements with 0 rows produced an error, and are now skipped.
- Corrected warning in ICESDatsusc() where NMDBiotic v3 did not result in a warning.
- Added the ICESBioic format 1.6 that includes the new GeneticPopulationCode field.
- Fixed bug in FilterLanding() by aadding expandFilterExpressionList().
- Fixed more bugs in writeLevel() and exported sanitizeExpression.
- Added parameter format class “folderPath”.
- Added branch “testing” which will deploy to the testingRepo.
- Structured the check-full.yaml.
- Fixed issue with parsing some valid variants of the xml landings format (landingerv2)
- Fixed issue with parsing some valid variants of the InterCatch Exchange format
- Added functions for extracting some standard tables from NMD biotic
- Added vignette for guiding data users at IMR to locate and parse biological samples.
- Moved functions for setting precision to RstoxFramework, which actually assumed an sp object.
- Changed definition of CatchFractionNumber and SampleNumber to numeric instead of integer, as this was the result from runProcess() used in runModel().
- Added comments to ‘stox-biotic-final-phase.csv’.
- Fixed bug in getNumberOfCores() where the number of cores was not restricted by the number of available cores.
- Introducing to skip versions after per-releases to avoid R from considering the pre-releases as newer than the release.
- Added better warning when at least one of bottomdepthstart and bottomdepthstop are missing.
- Improved warning when variables that cannot be converted to numeric as requested by the XSD are set to NA in ReadAcoustic()/ReadBiotic().
- Fixed bug when translating with PreserveClass = FALSE, in which case the class change could corrupt the condition of the translation (e.g., translating a copy of the DateTime in the Log table of the StoxAcousticData would convert that to string after the first line of the TranslationTable, and then any time condition will fail, as the class is no longer POSIX).
Summary
- This release contains improvements and bug-fixes in the ICESDatras() and the RegroupLengthICESDatras() functions.
Genaral changes
- Updated documentation of StoxAcoustic and StoxBiotic with tables defining each variable.
- Removed variable names related to the NMCBiotic tables “prey”, “preylengthfrequencytable” and “copepodedevstagefrequencytable”.
- Added unit specification of StoxAcoustic and StoxBiotic variables.
- Changed RegroupLengthICESDatras() to regroup lengths both in the HL and the CA table, and also to support recalculating both HLNoAtLngt and CANoAtLngt. Also added the parameters ResolutionTableVariables and ResolutionTable to support specices specific (or other variables) regrouping.
Bug fixes
- Fixed bug in RegroupLengthICESDatras(), where the columns were reordered with ResolutionTableVariables first.
- Fixed bug in ICESDatras() where distance was first rounded in nautical miles and then multiplied by 1852.
- Fixed bug in translateOneTranslationOneTable() used by Translate-functions, where type conversion was applied before applying the translation, which for a function such as IndividualAge > 9 resulted in 10, 11, … to be comared as text and thus not translated.
Warnings
- Fixed snprintf warnnings.
- Added a warning if there are more than one tag for at least one individual in AddToStoxBiotic() for NMDBiotic >= 3 files.
- Corrected warning ” There are more than one ‘serialnumber’ …” to end with “More than one serialnumber for the following cruise/station (of the fishstation table of the BioticData):” instead of “Duplicated serialnumber for the following cruise/station (of the fishstation table of the BioticData):”.
- Corrected warning for more NASC in B than in P.
- Corrected warninig for non-supported NMDEhcosounder format from >= 1.4 to >= 1.1.
- Fixed bug in RegroupLengthICESDatras(), where the columns were reordered with ResolutionTableVariables first.
- Added a warning if there are more than one tag for at least one individual in AddToStoxBiotic() for NMDBiotic >= 3 files.
- Removed variable names related to the NMCBiotic tables “prey”, “preylengthfrequencytable” and “copepodedevstagefrequencytable”.
- Fixed bug in ICESDatras() where distance was first rounded in nautical miles and then multiplied by 1852.
- Updated documentation of StoxAcoustic and StoxBiotic with tables defining each variable.
- Added unit specification of StoxAcoustic and StoxBiotic variables.
- Removed error when filtering BioticData read from ICESBiotic and ICESAcoustic XML.
- Fixed snprintf warnnings.
- Corrected warning ” There are more than one ‘serialnumber’ …” to end with “More than one serialnumber for the following cruise/station (of the fishstation table of the BioticData):” instead of “Duplicated serialnumber for the following cruise/station (of the fishstation table of the BioticData):”.
- Corrected warning for more NASC in B than in P.
- Corrected warninig for non-supported NMDEhcosounder format from >= 1.4 to >= 1.1.
- Removed the StoX XML from https://acoustic.ices.dk/submissions.
- Fixed bug in translateOneTranslationOneTable() used by Translate-functions, where type conversion was applied before applying the translation, which for a function such as IndividualAge > 9 resulted in 10, 11, … to be comared as text and thus not translated.
- Changed HaulNo to use the serialnumber and not the station variable of NMDBiotic >= 3 in ICESDatras().
- Changed RegroupLengthICESDatras() to regroup lengths both in the HL and the CA table, and also to support recalculating both HLNoAtLngt and CANoAtLngt. Also added the parameters ResolutionTableVariables and ResolutionTable to support specices specific (or other variables) regrouping.
- Changed error to warning when FilterUpwards from the top table.
- Added support for the string “NA” in WriteICESAcoustic() (used in the field DataProcessingTriwaveCorrection) (and also in WriteICESBiotic()).
- Disabled removing namespace prefix in readXmlFile(), which used xslt::xml_xslt().
- Added support for NMDEchosounder files in ICESAcoustic().
- Added defaults for GroupingVariables and AggregationVariables in RegroupLengthICESDatras().
- Reorderd TranslationTable to after VariableName, Conditional and ConditionalVariableNames in Translate functions.
- Moved do.call_robust() from RstoxFramework to RstoxData.
- Fixed bug with filterpropagation on branched data formats. E.g. agereadings not being removed when corresponding individuals where removed in biotic, and clarified documentation (#241, #240).
- Fixed bug when RstoxData functions relying on the global variable xsdObjects was called without RstoxData being loaded (“263”)
- Fixed bug with reading some logbook-variants (#237)
- Added CopyBiotic, CopyStoxBiotic, CopyICESBiotic, CopyICESDatras, CopyAcoustic, CopyStoxAcoustic, CopyICESAcoustic, CopyLanding and CopyStoxLanding, used for copying one column to another (possibly existing) column.
- Fixed bug in Translate functions where PreserveClass = TRUE had no effect.
- Updated documentation of DefineTranslation and the Translate functions.
- Added drop-down lists for the parameters VariableName and ConditionalVariableNames, and for valueColumn, newValueColumn and conditionalValueColumns in the case that the table is read from a file.
- Fixed bug in StoxBiotic() from NMDBiotic <= 1.4 files, where platform from the mission table was used as CatchPlatform. Changed to using the platform from the fishstation.
- Added support for adding variables in AddToStoxBiotic() that originate from CopyBiotic().
- Fixed bug in Translate functions, where PreserveClass = FALSE did not work.
- Changed check-full.yaml to run for ‘release’ and specific R versions (4.2, 4.1, etc).
- Unified how lengtht code is calculated for ICESBiotic and ICESDatras.
- Added RegroupLengthICESDatras() (replacing PrepareWriteICESDatras()), which regroups the lengths of the HL table of ICESDatras.
- Added lengthResolutionTable and lengthCode_unit_table to Definitions.R.
- Fixed bug in pkgdown.yaml.
- Added support for pre-releases, which are not deployed to the StoX repo.
- Added the function PrepareWriteICESDatras().
- Fixed typo in ICESDatras() (BycSpecRecCode changed to BySpecRecCode).
- Added specialstage in Maturity in CA.
- Added the LiverWeight in CA, with values from the liverweight in NMDBiotic
- Fixed typo in warning for more sa in B than in P.
- Added a unique for the Log table in StoxAcoustic for ICESAcoutsic.
- Added units for area number density, and one for megaton.
- Added the exported hasUnit().
- Fixed bug in ReadAcoustic when an ICESAcoustic xml files contains more than one instrument.
- Fixed bug with R < 4.2, where a filter process with unspecified FilterExpression retuns error “zero-length inputs cannot be mixed with those of non-zero length”. The error is returned both when opening the FilterExpression and when running the process.
- Pre-release before 1.7.1. Errors are expected.
- Added warning when ch_type P is missing or represent less sa than B.
- Pre-release before 1.7.1. Errors are expected.
- Added the exported functions WriteAcoustic() and WriteBiotic().
- Changed error to warning when type is not convertible in writeXmlFile().
- Improved warnings in StoxBiotic() when missing values are generated for different producttype etc.
- Cleaned up use of xsdObjects, which is now always used directly and not via RstoxData::xsdObjects.
- Added applyFunctionArgumentHierarchy() for use in RstoxFramework and in any other Rstox packages in need for evaluating the function argument hierarchy.
- Added doc for WriteAcoustic, WriteBiotic, FunctionArgumentHierarchy.
- Renamed match_arg() to match_arg_informative().
- Added warning for duplicated station in NMDBiotic, which leads to more than one Haul per Station. This is not supported when assigning Hauls in the map, where all Hauls of a Station are selected. Filtering out Hauls can be a solution.
- Exported printErrorIDs().
- Improved warnings AcousticDataToICESAcousticOne() when values are not found in ICES reference tables.
- Start of using semantic versioning (https://semver.org/). Before this release the two first version numbers represented the major and minor release number, in accordance with semantic versioning, whereas the third version number identified test versions. The major and minor releases (versions ending with 0.0 or 0) were considered as official versions. From this release and onwards, the third version number will represent patches (bug fixes), and are to be considered equally official as the major and minor releases. In fact, as patches are restricted to fixing bugs and not adding new functionality, the latest patch will be the recommended version.
- Fixed bug in getLogKey_ICESAcoustic().
- Disabled warning in Translate-functions when a table contained some but not all of the variables of the Translation.
- Added warning when SampleCount is used instead of the new SampleNumber in a filter, asking the user to change the filter.
- Updated the ICES XSDs.
- Restored the LogKey to the form where seconds are dropped in StoxAcousticData from ICESAcoustic files with minute resolution, to avoid issues with processData in existing StoX projects.
- Removed LogKey and EDSU from AcousticData read from ICESAcoustic files, which were added by reference in StoxAcoustic(). Added data.table::copy to fix this.
- Added test for allowed file extensions in readXmlFile() to avoid R crashing when e.g. RData file is accidentally used.
- Fixed temporary bug where StoxTimeZone was used from Definitions.R in processBioticData.R when it is not defined before onLoad.
- Fixed bug in the JSON schema of the Translation process data, where number, string and boolean were allowed for the NewValue field, in that order, whereas string and null is correct.
- Fixed bug in as.POSIXct_ICESAcoustic() where the minute resoslution was tested first, causing loss of available seconds.
- Fixed bug in StoxAcoustic() where LogKey in ICESAcoustic data was misspecified when using time format “YYYY-MM-DDThh:mm” or “YYYY-MM-DD hh:mm” (without seconds). The seconds (00) are now included.
- Fixed bug in StoxBiotic() where the time format “YYYY-MM-DDThh:mm” in an ICESBiotic xml files was truncated to only the date.
- Added exported functions MergeAcoustic(), unMergeAcoustic() and unMergeBiotic().
- Added exported function setColumnClasses().
Replaced the logical AddToLowestTable by the string SplitTableAllocation in AddToStoxBiotic(), allowing for allocating variables to either the default, highest or lowest table when splitting tables StoxBiotic.
-
Removed hard coded values for the following variables on ICESDatras() (variable name -> new value):
Table HH Country -> nation Ship -> platformname SweepLngt -> NA GearEx -> NA DayNight -> NA StatRec -> area + location (concatenation) HaulVal -> NA Distance -> distance (in meters) *GroundSpeed -> vesselspeed
Table HL SpecVal -> NA LenMeasType -> lengthmeasurement
Table CA Maturity -> NA MaturityScale -> NA AgeSource -> agingstructure OtGrading -> readability (only if agingstructure is 2) *PlusGr -> NA
Removed hard coded values for the following variables on ICESBiotic(): Platform -> platformname Validity -> NA *StatisticalRectangle -> area + location
- Fixed bug in DateTime in StoxBioticData, where milliseconds were pasted twice if present in the input data.
- Fixed bug reported in https://jira.imr.no/browse/STOX-544, occurring when splitting catchsample into SpeciesCategory and Sample, by unique() in firstPhase().
- Added exported function match_arg(). Modified writeXmlFile.R and added WriteAcoustic().
- Improved warnings when catchproducttype, sampleproducttype, individualproducttype or lengthmeasurement does not have the appropriate value.
- Added the parameter AddToLowestTable in AddToStoxBiotic(), which can be used for adding variables from tables in NMDBiotic or ICESBiotic that are split into two tables in StoxBiotic (fishstation and catchsample in NMDBiotic and Haul and Catch in ICESBiotic). When these tables are split into two tables StoX decides which variable should be placed in each table. E.g., geographical position is placed in the Station table of StoxBiotic, which implies that only the first position of several hauls that comprise one Station is kept. If one needs all positions, AddToLowestTable can be set to TRUE so that the positions are placed in the Haul table instead of the Station table of StoxBiotic.
- Added warning when at least one of bottomdepthstart and bottomdepthstop are missing, so that BottomDepth is NA.
- Included more informative warning when e.g. product types are not the required value in StoxBiotic().
- Updated to the new ICESAcoustic and ICESBiotic format (released March 2022).
- Increased significant digits of small numbers foomo 6 to 12.
- Relaxed the warning for translation columns not present in the data to accept if all are not present (unaffected table).
- Moved application of setRstoxPrecisionLevel() from each StoX function to RstoxFramework::runProcess().
- Added SpatialPolygonsDataFrame and matrix to setRstoxPrecisionLevel().
- Added the parameters TranslationDefinition, TranslationTable, VariableName, Conditional and ConditionalVariableNames to all Translate functions, specifically TranslateAcoustic(), TranslateBiotic(), TranslateICESAcoustic(), TranslateICESBiotic(), TranslateICESDatras(), TranslateLanding(), TranslateStoxAcoustic(), TranslateStoxBiotic() and TranslateStoxLanding(). This allows for specifying the Translation as a table in the Translate function, without the need for DefineTranslation(). DefineTranslation() can still be used, and must be used if reading the Translation from a file.
- Added sanitizer for DefineTranlation(), as the Translation process data is evaluated in Translate functions.
- Postponed ICESDatras_New().
- Changed the process data Translation from a table with columns VariableName, Value, NewValue, ConditionalVariableName, ConditionalValueColumn, to a table of the variable to translate in the first column; the column NewValue giving the values to translate to in the second column; followed by zero or more conditional variables. This supports multiple conditional variables, but restricts to translating only one variable at the time (although the old table i still supported, but cannot be generated in the GUI).
- Added support for NAs and other values in the same Translation process data.
- Added support for specifying a function as a string in Translation process data, usefull e.g. for setting fish larger than som value to mature.
- Renamed ConditionalVariableName to ConditionalVariableNames and ConditionalValueColumn to ConditionalValueColumns, as multiple values are now supported.
- Fixed bug in ICESDatras() occurring when there were rows with the same aphia and species, but with missing sex.
- Accepting NAs in convAgeSource().
- Added the functions FilterICESAcoustic(), FilterICESBiotic(), FilterICESDatras() and TranslateICESDatras().
- Changed the output of ICESBiotic(), ICESDatras() and ICESAcoustic() to combine tables from the different files in the same way that StoxBiotic() and StoxAcoustic() does. This also affects the name of the output of WriteICESAcoustic(), WriteICESBiotic() and WriteICESDatras(), which is no longer named by the input file to the Read* function.
- Added warning when a filter species a table that does not exist in the data.
- Fixed bug where log-distances of duplicated LogKey (sometimes occurring when times are on minutes instead of seconds resolution) were not properly removed in all tables.
- Removed unwanted and in some cases erroneous hard coded values in ICESDatras(), as listed in the following table:
- Gear: Changed from “GOV” to the NMDBiotic/fishstation variable gear
- DoorType: Changed from “P” to the NMDBiotic/fishstation variable trawldoortype
- DoorSurface: Changed from 4 to the NMDBiotic/fishstation variable trawldoorarea = 4.5,
- DoorWgt: Changed from 1075 to the NMDBiotic/fishstation variable trawldoorweight = 1075
- KiteDim: Changed from 0.8 to NA_real_, as there is no relevant information in NMDBiotic
- lngtCode: Changed from interpreting herring/sprat and crustatians to using RstoxData:::getLengthCodeICES()
- Removed all filtering by species. This should rather be done in FilterBiotic() or rather FilterICESDatras()
Added warnings for when (catch)producttype != 1, (sample)producttype != 1, (individual)producttype != 1, or lengthmeasurement != ‘E’.
Added support for NMDBiotic files of mixed version (<= and > 1.4) in AddToStoxBiotic() (removing the prey table and other unused tables, as consistent link to the individual table is not provided by the XML schema.).
Fixed bug causing stream = TRUE to fail on MacOS Monterey in readXmlFile().
- Hiding the parameters VariableName and ConditionalVariableName when DefinitionMethod = “Table” in Translation().
- Renamed CatchFractionCount to CatchFractionNumber and SampleCount to SampleNumber in StoxBiotic. CatchFractionCount was considered misleading, as this variable is often not a result of counting but rather an estimate from total and sampled weight.
- Changed type of the columns of the Translation process data to accept strings, numeric and boolean (preivously restricted to string).
- Changed TranslationTable to Table in DefineTranslation(). Fixed bug where NA in Tanslation was not converted properly to the type of the existing data. Added change of class in ICESBiotic() as per the XSD. Fixed bug in AddToStoxBiotic(), where variables from agedetermination in NMDBiotic >= 3 were not added. Code changed to use the xsd to determine the variables that can be added.
- Final version for the release of StoX 3.2.0.
- Added warning when ValueColumn, NewValueColumn or ConditionalValueColumn did not exist in the file in DefineTranslation().
- Added return value from setorderv_numeric().
- Reverted to the original createOrderKey() of StoX 3.1.0, used in setorderv_numeric() and further in RstoxBase::formatOutput(), in order to produce the same seeds in RstoxBase::ImputeSuperIndividuals().
- Added parameters VariableName, ConditionalVariableName and ConditionalValueColumn to DefineTranslation(), to support full flexibility of column names in the resource file. Also added the parameter PreserveClass to Translate* functions, specifying whether to allow for the translation to change class of the data, e.g. form integer to string. Specified NAs in ICESBiotic() to the class defined by ICES.
- Fixed bug in readVariableTranslation() (where eval() was used instead of get()).
- Added warning when adding a variable that already exists in AddToStoxBiotic(), particularly aimed at SpeciesCategory in ICESBiotic, which has a different meaning that SpeciesCategory in StoxBioticData.
- Added the parameters ValueColumn and NewValueColumn to DefineTranslaion().
- Added CompensateEffectiveTowDistanceForFishingDepthCount() for NMDBiotic data with hauls made at several depths.
- Fixed error in links to documentation in RstoxData.
- Final version for the release of StoX 3.2.0.
- Final version for the release of StoX 3.2.0.
- Added class to stox-biotic-final-phase.csv, used when generating an empty table in StoxBiotic().
- Added support for empty tables in StoxBiotic() and in setPrecisionLevelOneDT().
- Refactored StoxBiotic() for ICESBiotic xml files with NumberAtLength given.
- Fixed formatting of output from WriteICESBiotic() so that precision is kept and values are not padded with blanks and zeros. Fixed bug in writeXmlFile().
- Fixed bug where TranslateStoxBiotic() and similar functions changed type of the data, so that translating numeric values did not work properly.
- Changed Distance in ICESBiotic() to distance * 1852.
- Fix bundled miniz to not using unaligned access. This should fix CRAN’s gcc-UBSAN check.
- Bump miniz and pugixml to the latest release version.
- Fixed bug where TranslateStoxBiotic() and similar functions changed type of the data, so that translating numeric values did not work properly.
- Final version for the release of StoX 3.1.0.
- Fixed bug in StoxBiotic(), where date and time were borrowed from Station to Haul, which could crash due to missing values in StationKey.
- Removed interpretation of agingstructure in ICESBiotic().
- Changed to sort in en_US_POSIX-locale in createOrderKey() using the stringi-package, which ensures platform independence while replicating sorting done by data.table.
- Fixed bugs and added auto-detect xsd for reading zipped xml files.
- Fixed bug in StoxAcoustic for ICESAcoustic data, where log-distances with no acoustic records were deleted.
- Changed to sort in C-locale in createOrderKey() using the stringi-package, to comply with data.table’s philosophy of platform independence.
- Added sanitizeFilter() to avoid system calls in filter.
- Removed hard coded conversions in ICESBiotic(), moving the responsibility of such conversions to the translation functions.
- Optimized createOrderKey() for faster execution.
- Renamed ReportICESAcoustic(), ReportICESBiotic() and ReportICESDatras() to WriteICESAcoustic(), WriteICESBiotic() and WriteICESDatras(), respectievly.
- Added TranslateICESAcoustic() and TranslateICESBiotic().
- Added option of a conditional variable in DefineTranslation() and Translate*().
- Removed maturity conversion in ICESBiotic().
- Fixed format of columns of ReportICESBiotic().
- Fixed time format of StoxAcoustic().
- Moved translation of ICESBiotic before merging levels as merging changes name of some variables.
- Changed to keep original FishID and add sequetial integers for individuals regenerated from Catch continuing from the maximum FishID.
- Added documentation of the StoxBiotic format.
- Fixed bug in LengthResolution for ICESBiotic where only the first value was used.
- Fixed bug when converting length for ICESBiotic, where values were multiplied by 100 instead of 10 from mm to cm.
- Changed TowDistance to nautical miles.
-
NEW: Reading XML files with namespace prefix is now supported.
-
NEW: Writing XML files (alpha) is now supported.
- Fixed bug in DateTime for ICESAcoustic files, and bug in translateOneTable() causing incomplete translation in StoxAcoustic from ICESAcoustic files.
- Added ChannelDepthUpper-ChannelDepthLower as Channel in StoxAcousic.
- Fixed bug in createOrderKey() where columns that are non-convertable to nunmeric were replaced by NA instead of being left unchanged
- Fixed bug in
filterData, where propagateUpwards = TRUE did not remove rows of the higher tables if these rows were not present in the filtered table.
- Corrected type of variables of
StoxAcoustic for NMDEchosounder input xml files.
- Added possible values for
redefinitionTable.
- Renamed
readVariableConversion() to readVariableTranslation().
- Refactored
translateVariables().
- Throw error when file is missing in
readXmlFile().
- Refactor some of the reading functions to get rid of
readr dependency.
- New feature: ECA integration. This is merged some time ago.
- Final version for the release of StoX 3.0.0.
- Renamed TowedDistance to TowDistance and EffectiveTowedDistance to EffectiveTowDistance.
- Interpret keys as numeric (possibly separated by slash) if possiible when ordering StoxAcoustic and StoxBiotic.
- Added support for NMDBiotoic1.4 and NMDBiotoic1.1 in StoxBiotic().
- Added DefineTranslation and TranslateAcoustic, TranslateBiotic, TranslateLanding and TranslateStoxLanding.
- Removed all Convert-functions. These may be added later.
- Added/fixed FilterLanding and FilterStoxLanding.
- Added all agedetermination variables in BioticData2GeneralSamplingHierarchy(), making these available for AddToStoxBiotic.
-
ICESDatras: remove reference to SurveyName and addSurveyType parameters.
- Add reporting functions for ICES exports (
ReportICES*()).
- Add Intercatch data parser (
parseInterCatch()).
- Add
roundDrop0() to replace round() as a more robust rounding function.
-
getICESShipCode: improve ship conversion by removing the deprecated entries and sorting.
-
ICESDatras: remove reference to SurveyName parameter.
- Move
readr to suggests.
- Made
compareICES() robust to missing internet connection.
-
ICESDatras returning matrix to be written as csv by RstoxFramework::runProcess.
- Cleaned up translations using vocabulary for ICES data.
- Fixed bug with DateTime in
StoxAcoustic().
- Added
backwardCompatibility and removed NumberOfCores.
- Refactored the functions for writing ICESBiotic and ICESAcoustic files to include NMDBioticToICESBiotic() and to use similar methods.
- In
writeICESDatras(): Change country code ‘NOR’ to ‘NO’.
- Change the default NumberOfCores to 1L in all parallel-able functions.
- Refactor
WriteICESAcoustic(), WriteICESBiotic(), WriteICESDatras() into prepareICESAcoustic(), prepareICESBiotic(), prepareICESDatras().
- Tests: do not attempt to copy file outside
tempdir() in test-readXmlFile.R file.
- Github actions: Update
check-full.yaml file.
-
prepareICESAcoustic() to use data.table and not use format().
- Fix parallel
lapplyOnCores() behavior in Windows platform.
- Remove
StoxAcousticStartMiddleStopDateTime() function.
- Prepare for CRAN submission: Remove all attempts to modify the global environment.
- Prepare for CRAN submission: Shorten the package title.
- Update variables in
processDataSchema.json.
- Delete
zzz.R and pkgname.R and moved the contents to RstoxData-package.R.
- Added a
NEWS.md file to track changes to the package.
- Added a
cran-comments.md file for CRAN submission.
-
StoxExport functions are now writing output files in tempdir().
- Various small fixes for adhering to CRAN policies.
- Changed SampleKey to catchsampleid.
- StoxExport: Adopt the latest unified DATRAS format.
- StoxExport: Fix function for getting ICES ship code.
- stoxBioticObject: Fix DateTime column from ICES Biotic has a different class compared to the one from NMD Biotic.
- Documentation fix.
- Added support for undefined
signifDigits.
- Modified
setPrecisionLevelOneDT() to keep at least 6 significant digits.
- Removed unit in variable and parameter names.
- Added IDs separated by ‘-’.
- Fix small bug in
DefineDataTranslation().
- Added
Cruise in StoxAcoustic.
- Added
Cruise in the Cruise table of StoxBiotic.
- Added Redefine, Translate and Convert.
- Changed
AcousticCategory to character.
- Added warning for non-unique
LogKey.
- Added
AddStoxBioticVariables() and ConversionType in ConvertStoxBiotic.
- Add NMD Biotic format v3.1 support.
- Renamed Cores to NumberOfCores and LengthInterval to LengthIntervalCentimeters.
- Filter: Filtering upwards now able to skip empty tables.
- Minor update to stox acoustic.
- Minor update to stox acoustic.
- Minor update to stox acoustic.
New features: * Add multiple data export feature: 1. ICES acoustic XML format to ICES acoustic CSV format 2. NMD biotic v3 XML format to ICES biotic CSV format 3. NMD biotic V3 XML format to NS-IBTS ICES Datras (CSV) format
Bug fixes: * Fixes ICES Acoustic/Biotic vocabulary generation when parsing ICES’s XML files
New features: * Add filtering with upward propagation
Bugfixes: * Propagate down now able to skip empty tables (e.g., NMD v3 prey)
- Add define and update variables.
- More fixes in stox acoustic.
- Fixed bug with non-unique tables from ICESAcoustic.
New minor release. Notable new features are the support for NMD Biotic v1.4 and ICES Biotic data input.
Changelog: * Updated README * Increment version * Add test for StoxBiotic * Add ICES Biotic to StoXBiotic format conversion * readXmlFile: Fix typo * Add support for converting NMD Biotic v1.4 format to StoxBiotic
Changelog: * Update README(.md) * Increment version * Refresh xsdObjects data * Fix documentation * Add verbose output switch for reading XMLs * Add tests for StoxBiotic and StoxAcoustic functions * Fix merging proses in StoxAcoustic() after correction of the column types * Correct the data types in result tables * Fix StoxBiotic function
Updated XML reader, RstoxBiotic and RstoxAcoustic functions.
Changes: * Update DESCRIPTION * Appveyor needs the whole R version digit * Increment minor version * Update CIs to use only R 3.5 * Merge branch ‘biotic_acoustic’ of github.com:StoXProject/RstoxData into biotic_acoustic * Filter draft * XML stream read speed improvements * Result tables is now ordered * Suppress warning on utils data * metadata from readXmlFile is now a data.table * Update ICES XSDs raw and compiled data objects * Update stoxBioticObject data * Fix processBioticData length resolution converter * icesBiotic is now supported * Fix NMDBiotic v.1x xsdObjects * Added documentation for DataTypes * Merge branch ‘biotic_acoustic’ of github.com:StoXProject/RstoxData into biotic_acoustic * Added documentation for BioticData * StoxBiotic: Remove duplicate rows from SpeciesCategory table * Update data for conversion * Latitude and Latitude2 * Latest StoxBiotic updates * Merge branch ‘biotic_acoustic’ of github.com:StoXProject/RstoxData into biotic_acoustic * Minor fix * Fix (another) encoding problem * Fix documents * Update StoxBiotic conversion data and scripts * Clean up data.table warnings and inefficiencies * Removed PDFs * Add StoxAcoustic and StoxBiotic
- New minor release, various bug fixes.
- New minor release, mainly for various bug fixes.
- New Feature: Reading ICES acoustic XML files.
Changes: * Rebrand to RstoxData. * Add support for all biotic formats.