Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue porting page-blob to blockblob #116

Open
qootec opened this issue Nov 12, 2021 · 3 comments
Open

Issue porting page-blob to blockblob #116

qootec opened this issue Nov 12, 2021 · 3 comments

Comments

@qootec
Copy link

qootec commented Nov 12, 2021

Scenario:

  • Using Windows download for 0.6.20
  • I have a vhdx uploaded to Azure. By design, it became a page blob.
  • I planned to move it to archive, but page blobs can't be archived. I wasn't aware of that...
  • I now want to use blobporter to duplicate the blob to a blockblob and remove the pageblob afterwards
    (later I want to do that on several TB of VHDX that I've uploaded)

The command I tried (with some confidential info removed):

blobporter -f "https://saMyAccount.blob.core.windows.net/system20211012/Part1/Backup%202021-10-12%20073026/complexID.vhdx" -n Part1/Backup%202021-10-12%20073026/complexID.vhdx.removeExtension -c system20211012 -t blob-blockblob

Issue (with some confidential info removed):

Files to Transfer (blob-blockblob) :
Source: saMyAccount.blob.core.windows.net/system20211012/Part1/Backup 2021-10-12 073026/complexID.vhdx Size:306184192
2021/11/12 16:08:13 -> github.com/Azure/azure-storage-blob-go/azblob.NewResponseError, /home/travis/gopath/pkg/mod/github.com/!azure/[email protected]/azblob/zz_generated_response_error.go:29
= RESPONSE ERROR (ServiceCode=InvalidBlobType) =
Description=The blob type is invalid for this operation.
RequestId:975a66aa-301e-015d-40d7-d72bfe000000
Time:2021-11-12T15:08:13.1203140Z, Details:
Code: InvalidBlobType
PUT https://saMyAccount.blob.core.windows.net/system20211012/Part1/Backup 2021-10-12 073026/complexID.vhdx?blockid=MD...someid...3D&comp=block&timeout=61
Authorization: REDACTED
Content-Length: [0]
User-Agent: [BlobPorter/0.6.20/amd64/Azure-SDK-For-Go/0.0.0 azblob/2018-03-28 Azure-Storage/0.5 (go1.11; Windows_NT)]
X-Ms-Client-Request-Id: [ad501250-2412-481d-6b73-cdcc3ddcc887]
X-Ms-Copy-Source: [saMyAccount.blob.core.windows.net/system20211012/Part1/Backup 2021-10-12 073026/complexID.vhdx?se=2021-11-12t21%3A08%3A12z&sig=REDACTED&sp=r&sr=b&sv=2018-03-28]
X-Ms-Date: [Fri, 12 Nov 2021 15:08:12 GMT]
X-Ms-Source-Range: [bytes=260046848-268435455]
X-Ms-Version: [2018-03-28]

RESPONSE Status: 409 The blob type is invalid for this operation.
Content-Length: [228]
Content-Type: [application/xml]
Date: [Fri, 12 Nov 2021 15:08:12 GMT]
Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
X-Ms-Error-Code: [InvalidBlobType]
X-Ms-Request-Id: [975a66aa-301e-015d-40d7-d72bfe000000]
X-Ms-Version: [2018-03-28]

Analysis:

Any suggestions?
Thanks,
Johan

@qootec
Copy link
Author

qootec commented Nov 12, 2021

For your information: I've worked around the issue using "azcopy" instead of blobporter.
That solution seems to work fine.

@N1kolayS
Copy link

Hi all!
I have a same error.
I tried upload large file up 45Gb
blobporter -c fastq -f WGS-B-SDD130519-240402_2.fastq.gz

And i got same error

`BlobPorter 
Copyright (c) Microsoft Corporation. 
Version: 0.6.20
---------------

Files to Transfer (file-blockblob) :
Source: WGS-B-SDD130519-240402_2.fastq.gz Size:47264358266 
2021/11/18 09:10:57 -> github.com/Azure/azure-storage-blob-go/azblob.NewResponseError, /home/travis/gopath/pkg/mod/github.com/!azure/[email protected]/azblob/zz_generated_response_error.go:29
===== RESPONSE ERROR (ServiceCode=InvalidBlobOrBlock) =====
Description=The specified blob or block content is invalid.
RequestId:c427b903-a01e-00a3-4043-dc8f6b000000
Time:2021-11-18T06:10:57.5029439Z, Details: 
   Code: InvalidBlobOrBlock
   PUT https://wgs.blob.core.windows.net/fastq/WGS-B-SDD130519-240402_2.fastq.gz?blockid=removed_for_secure&comp=block&timeout=61
   Authorization: REDACTED
   Content-Length: [8388608]
   User-Agent: [BlobPorter/0.6.20/amd64/Azure-SDK-For-Go/0.0.0 azblob/2018-03-28 Azure-Storage/0.5 (go1.11; linux)]
   X-Ms-Client-Request-Id: [569f3fbf-5476-4703-65ee-1720f05cef9f]
   X-Ms-Date: [Thu, 18 Nov 2021 06:10:54 GMT]
   X-Ms-Version: [2018-03-28]
   --------------------------------------------------------------------------------
   RESPONSE Status: 400 The specified blob or block content is invalid.
   Content-Length: [234]
   Content-Type: [application/xml]
   Date: [Thu, 18 Nov 2021 06:10:56 GMT]
   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
   X-Ms-Error-Code: [InvalidBlobOrBlock]
   X-Ms-Request-Id: [c427b903-a01e-00a3-4043-dc8f6b000000]
   X-Ms-Version: [2018-03-28]
`

@forsythg
Copy link

I've also had a similar issue, but in my case it's documentation related. Long story-short, it appears converting from page blobs to block blobs is only possible within the same storage account and for a whole container from one container to another:

Example - this works:

$env:ACCOUNT_NAME="<account A name>"
.\BlobPorter.exe -f "https://<account A name>.blob.core.windows.net/srccontainer" -c destcontainer -t blob-blockblob```

Example - this fails:

$env:ACCOUNT_NAME="<account B name>"
$env:SRC_ACCOUNT_KEY="abc...."
.\BlobPorter.exe -f "https://<account A name>.blob.core.windows.net/srccontainer" -c destcontainer -t blob-blockblob```

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants