Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About Creating Database Issues #212

Closed
lxqaoliao opened this issue Apr 29, 2024 · 12 comments
Closed

About Creating Database Issues #212

lxqaoliao opened this issue Apr 29, 2024 · 12 comments

Comments

@lxqaoliao
Copy link

Excuse me, if I directly use the following case, I will report a null exception. What is the problem?
//Application specific root instance
Final DataRoot root=new DataRoot();

//Initialize a storage manager ("the database") with the given directory
Final Embedded Storage Manager storageManager=Embedded Storage. start(
Root,//root object
Paths. get ("data")//Storage directory
);
image

@hg-ms
Copy link
Contributor

hg-ms commented Apr 29, 2024

The exception is caused by a “null” root. This may be caused by a previous start of the storage with the root object not supplied or being null.
You can simply delete the data folder to solve that

or set the root explicitly:

final EmbeddedStorageManager storageManager = EmbeddedStorage.start(Paths.get("data"));
Object loadedRoot = storageManager.root();
if(loadedRoot == null) {
	System.out.println("Storage root is null");
	storageManager.setRoot(new DataRoot());
	storageManager.storeRoot();
}
final DataRoot root = (DataRoot) storageManager.root();

@lxqaoliao
Copy link
Author

Can data loading only be done through Lazy?

@hg-ms
Copy link
Contributor

hg-ms commented Apr 29, 2024

By default, the complete object graph (starting with the defined root) is loaded into memory when the storage starts.
Lazy loaded parts must be explicitly defined by using lazy references.
Please be aware that every lazy reference has some overhead (about 88 Bytes) and occupies memory even if the lazy referenced data is not loaded. It is (in most cases) more efficient to lazy reference lager structures like whole collection than many small objects.

@lxqaoliao
Copy link
Author

I always use it directly
Embedded Storage Manager storage=EmbeddedStorage.start (Paths. get (path));
Storage. start(); To retrieve data, every time it is retrieved, some files will be created in the storage directory. Can I not create it while reading? Or can it be read directly?

@hg-ms
Copy link
Contributor

hg-ms commented Apr 30, 2024

After the storage has started all data (except the lazy referenced data) is available in memory.
Accessing the data is done by plain java via the retrieved root object.
When a new storage is created it will persist some internal data, even if no user data is persisted.

Here is simple example that loads and stores some data:

public class SimpleExample {

	public static class MyRoot {
		private List<UUID> data = new ArrayList<>();
		
		public void add(UUID uuid) {
			this.data.add(uuid);
		}
		
		public List<UUID> getList() {
			return this.data;
		}
	}
	
	public static void main(String[] args) {
		
		//start storageManager
		EmbeddedStorageManager storage = EmbeddedStorage.start(Paths.get("SimpleExample"));
		
		//Init storage root if not yet done
		if(storage.root() == null) {
			storage.setRoot(new MyRoot());
			storage.storeRoot();
		}
		
		//get storage root object
		MyRoot myRoot = (MyRoot) storage.root();
		
		//read loaded data
		System.out.println("Data loaded: " + myRoot.getList());
		
		//add an item
		myRoot.add(UUID.randomUUID());
		
		//store the modified object
		storage.store(myRoot.getList());
		
		//shutdown storage
		storage.shutdown();
	}
	
}

@lxqaoliao
Copy link
Author

How can I avoid generating logs or other files every time I load data? Every time I load data from a certain directory, a file is generated, which takes up a lot of space

@hg-ms
Copy link
Contributor

hg-ms commented Jun 11, 2024

Can you please provide more details on the generated file like its name and location?

@lxqaoliao
Copy link
Author

In the channel_1 directory, some dat files will be generated when saving data. If saved repeatedly, the generated dat file will be about twice as large as before

@lxqaoliao
Copy link
Author

If possible, can I delete all the original files before storing Data and then re store Data?

@hg-ms
Copy link
Contributor

hg-ms commented Jun 11, 2024

The “.dat” files contain the stored data. Every store appends its data to those files.

Deleting an object is done by removing all references to it and persisting those modifications. The deleted data will be removed from the storage files by the storage garbage collector later, see Housekeeping for details.

//deleting the first element by removing all references and storing the modification
myRoot.getList().remove(0);
storage.store(myRoot.getList());

If you want to delete an old storage completely you can delete the whole storage directory (the one that contains the channel_x directories) and all its content only if the storage is shut down. Do not delete the “.dat” files only, this will corrupt the storage.

@lxqaoliao
Copy link
Author

Is it true that creating it takes a lot of time?

@lxqaoliao
Copy link
Author

EmbeddedStorageFoundation<?> foundation = EmbeddedStorageConfiguration.Builder()
.setStorageDirectory("E:\test\swat\nodes")
.setDataFileMinimumUseRatio(1.0)
.createEmbeddedStorageFoundation();

    foundation.onConnectionFoundation(BinaryHandlersJDK17::registerJDK17TypeHandlers);
    final EmbeddedStorageManager storage = foundation.createEmbeddedStorageManager().start();
    //start storageManager

    //Init storage root if not yet done
    if(storage.root() == null) {
        storage.setRoot(new MyRoot());
        storage.storeRoot();
    }

    //get storage root object
    Nodes myRoot = (Nodes) storage.root();

    //shutdown storage
    storage.shutdown();

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants