Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimize sandbox directory structure #38

Merged
merged 2 commits into from
Jul 10, 2023
Merged

Conversation

Vanient
Copy link
Member

@Vanient Vanient commented Jun 30, 2023

  1. As we just need to mount guest needed directory and files to guest, other files like console sock and sandbox.json should not be mounted into guest, so we create a shared directory to store those needed by guest and mount it only.

after optimizing:
image

  1. create bundle directory earlier
    Before this, container bundle directory was initialized and created in spec handler, but it was used by storage handler and io handler which before spec handler, bundle directory was still "" in storage handler and io handler, which caused files were created under "/". So we should initialize and create bundle directory in metadata add handler which is before all other ones.

Copy link
Member

@flyflypeng flyflypeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

let bundle = format!("{}/{}", sandbox.base_dir, self.container_id);
let bundle = format!(
"{}/{}/{}/{}",
sandbox.base_dir, "shared", self.container_id, "bundle"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shoud we define some const variables to replace "shared" and ''bundle"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's better

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed @Burning1020

@@ -118,7 +118,7 @@ impl ContainerFactory<KuasarContainer> for KuasarFactory {
req: &CreateTaskRequest,
) -> containerd_shim::Result<KuasarContainer> {
rescan_pci_bus().await?;
let bundle = format!("{}/{}", KUASAR_STATE_DIR, req.id);
let bundle = format!("{}{}/{}", KUASAR_STATE_DIR, req.id, "bundle");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you miss a '/' ?
format!("{}/{}/{}", KUASAR_STATE_DIR, req.id, "bundle");

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because KUASAR_STATE_DIR is defined as "/run/kuasar/state/", with the last '/', so i deleted '/' here......
But maybe it should not ended with ‘/’, I'll optimize it together.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed @Burning1020

@Vanient Vanient force-pushed the main branch 2 times, most recently from 2654e8e to a1f4c5a Compare July 5, 2023 01:20
Copy link
Member

@Burning1020 Burning1020 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -118,7 +118,7 @@ impl ContainerFactory<KuasarContainer> for KuasarFactory {
req: &CreateTaskRequest,
) -> containerd_shim::Result<KuasarContainer> {
rescan_pci_bus().await?;
let bundle = format!("{}/{}", KUASAR_STATE_DIR, req.id);
let bundle = format!("{}{}/{}", KUASAR_STATE_DIR, req.id, "bundle");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we assume that the bundle is only path that the task server can get, I think maybe it is enough to leave the path what it is, without a "bundle" subpath.

) -> Result<()> {
if device_id.is_some() {
self.vm.hot_detach(&device_id.unwrap()).await?;
} else if fs_type == "bind" {
let mount_point = format!("{}/{}", self.base_dir, id);
let mount_point = format!("{}/{}/{}/{}", self.base_dir, "shared", container_id, id);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the storage is in the scope of pod( or sandbox), we can not mount the storage to a specific container path.

@@ -158,7 +158,10 @@ where
} else {
m.source.clone()
};
let host_dest = format!("{}/{}", self.base_dir, &storage_id);
let host_dest = format!(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the storage is in the scope of pod( or sandbox), we can not mount the storage to a specific container path.

@@ -186,7 +189,7 @@ where
driver_options: vec![],
fstype: "bind".to_string(),
options: vec![],
mount_point: format!("{}{}", KUASAR_STATE_DIR, &storage_id),
mount_point: format!("{}{}/{}", KUASAR_STATE_DIR, &container_id, &storage_id),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the storage is in the scope of pod( or sandbox), we can not mount the storage to a specific container path.

@@ -264,15 +270,15 @@ where
Ok(())
}

async fn gc_storages(&mut self) -> Result<()> {
async fn gc_storages(&mut self, container_id: &str) -> Result<()> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe this container_id param is not necessary if we do not relate any storage to a specific container, storage is in the scope of sandbox.

As we just need to mount guest needed directory and files to guest, other
files like console sock and sandbox.json should not be mounted into
guest, so we create a shared directory to store those needed by guest and
mount it only.

after optimizing:
/kuasar/d691426a7c2407fee8de3a09aca38e282acffde838fd001d1b2416ebf834d947/
├── console.sock
├── sandbox-d691426a7c2407fee8de3a09aca38e282acffde838fd001d1b2416ebf834d947.log
├── sandbox-d691426a7c2407fee8de3a09aca38e282acffde838fd001d1b2416ebf834d947.pid
├── sandbox.json
├── shared
│   ├── 677c691aa0ca843fc3131458512fea56de1fbf01983e7478e60a4e0a28a31108
│   │   ├── config.json
│   │   ├── init.pid
│   │   └── log.json
│   ├── storage1
│   │   ├── bin
│   │   ├── dev
│   │   ├── etc
│   │   ├── home
│   │   ├── proc
│   │   ├── root
│   │   ├── sys
│   │   ├── tmp
│   │   ├── usr
│   │   └── var
│   ├── storage10
│   ├── storage8
│   └── storage9
├── virtiofs.log
└── virtiofs.sock

Signed-off-by: Vanient <[email protected]>
Before this, container bundle directory was initialized and created in
spec handler, but it was used by storage handler and io handler which
before spec handler, bundle directory was still "" in storage handler
and io handler, which caused files were created under "/".
So we should initialize and create bundle directory in metadata add
handler which is before all other ones.

Signed-off-by: Vanient <[email protected]>
@abel-von
Copy link
Contributor

LGTM

@abel-von abel-von merged commit 90d1c40 into kuasar-io:main Jul 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants