Support New Database: IoTDB #226
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
init repo. make (using mingw32-make) success. The binary is executable on windows.
Init iotdb, part Query not completed 1. pkg\query\iotdb.go added, but not completed yet 2. cmd\...\iotdb\common.go added, but not completed yet 3. cmd\...\iotdb\devops.go added, but not completed yet 4. go.mod and go.sum modified
1. complete devops.go. without devops_test.go Other files are not completed so it may cannot generate queries till now. 2. fix some typo in internal\utils\time_interval.go
1. devops query generate completed. 2. test file added, but NOT completed
queries test complete
serializer complete
cmd/tsbs_load_iotdb
tsbs_load_iotdb is completed, but no unit tests added in this part
update iotdb-client-go version, use timeoutInMs
complete scan test 也许应该做更多努力来增强batch的性能
update some user friendly tips. timeout info, workers info.
Users can now specify timeout for session opening check
tsbs_run_queries_iotdb has been completed. Some CRTODO comments need to be removed
CRTODO removed
using back quote(`) in IP address instead of using quotation marks(")
using DATETIME-INPUT in data serializer timestamp in LONG INT is ambiguous
fix bug in GroupByTimeAndPrimaryTag should use one hour in `group by` instead of one minute
added documentation, readme for iotdb
1.using InsertRecords 2.serilaizer string without '' 3.using unixNano as timestamp in serializer output file, but should be converted into millisecond before insert. 4.readme updated 5.there is 3 lines in serializer output for each data point
using 4lines structure for serializer, not completed can insert data without tags. can NOT handle tags yet in this commit.
using 4lines serializer storage tags in attributes of node '_tags'
update workersNum warning in insertion test less than five --> less than 2
optimize ProcessBatch function using records struct to simplify code
fix typo parseDateToInterface -> parseDataToInterface
support csv export but readme is not updated
make sure csv export can be used offline if to-csv == true, then there is no need to connect to any database, because user want to generate csv files locally
updated docs and comments. added arguments check init bug fixed: forget to init ProcessedTagsDeviceIDMap in Init.
updated test cases for scan.go and serializer.go return empty string while there is no tags
correct SQL queries. support printing queries responses.
[important] Printing is not counted into query time. Do NOT send close signal after each query.
updated main readme
fix bug: query dataset did not close in all cases. Now dataset will always close after query, and will be traversed in query
check dataset close error before return
better error catch for dataset closing
updated SQL query
close sql dataset in insertion
Traverse query results in insertion
fix defer ds.Close() in insertion
use ExecuteNonQueryStatement for inserting tags
close dataSet in queries anyway
for GroupByTime use 1min， fix issues timescale#214 IoTDB is same with InfluxDB and TimescaleDB now。 And that meas the behavior of IoTDB single-group-by queries is NOT same with README, but is same with other databases.
User can choose TS type and whether to store tags now. Added flag `aligned-timeseries` and `store-tags` for tsbs_load_iotdb
after queries, not traverse query results
treat all integer as int32
do NOT treat all integer as int32
Fixed Bug, supported using BasicPath for data generation
Fix test cases TestSerialize_normal
Fix panic in timescale target with `iot` use case (timescale#233)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Support New Database: IoTDB
About IoTDB
Apache IoTDB (Database for Internet of Things) is an IoT native database with high performance for data management and analysis, deployable on the edge and the cloud. For more details about Apache IoTDB, please take a look at: https://iotdb.apache.org/
Supported test cases for IoTDB
Supported devops and cpu-only test cases for IoTDB now.
Supports multiple versions of IoTDB server: 0.13.x, 1.0.x, 1.1.x ...
Documentation
Documents are added, README is updated.
New features
Users can use various parameters to test the performance of IoTDB in different configurations.
Users can export data into csv files in format of IoTDB.
Tests
The necessary unit tests have been added. All tests are passed.
And these codes have used for multiple complete tests.