Delimited Text Reader Performances#
vtkDelimitedTextReader memory footprint is now much lower during computation.
(i.e. inside the RequestData, called from Update)
Also, speedup are expected (due to the drastic diminution of allocations)
While it used to store the whole data as string before doing the conversion to numeric values (if relevant) leading to data duplication at some point of the process, it now performs conversion on the fly.
There is no change once RequestData return, also no change in the output
vtkTable.
This is a memory reduction during the read process.
Early tests with a 100k lines and 9 columns csv file through heaptrack gives the following:
version |
peak memory |
peak contribution |
calls to allocation |
duration |
|---|---|---|---|---|
master |
58.5 MB |
41.9 MB from |
1,042,111 |
0.873 s |
this topic |
17.9 MB |
6.3 MB from |
39,784 |
0.344 s |