Ryfujk

Was it really inappropriate to write a pull request for the company I interviewed with?

Split a number into equal parts given the number of parts

Can an earth elemental drown/bury its opponent underground using earth glide?

Why did the Cray-1 have 8 parity bits per word?

School performs periodic password audits. Is my password compromised?

Giving a talk in my old university, how prominently should I tell students my salary?

Should we avoid writing fiction about historical events without extensive research?

If nine coins are tossed, what is the probability that the number of heads is even?

Is there a way to find out the age of climbing ropes?

PTIJ: Should I stay away from my computer?

Are there other characters in the Star Wars universe who had damaged bodies and needed to wear an outfit like Darth Vader?

PTIJ: Aharon, King of Egypt

1970s scifi/horror novel where protagonist is used by a crablike creature to feed its larvae, goes mad, and is defeated by retraumatising him

Called into a meeting and told we are being made redundant (laid off) and "not to share outside". Can I tell my partner?

How to mitigate "bandwagon attacking" from players?

How does insurance birth control work?

Is divide-by-zero a security vulnerability?

How do we objectively assess if a dialogue sounds unnatural or cringy?

I encountered my boss during an on-site interview at another company. Should I bring it up when seeing him next time?

Why do phishing e-mails use faked e-mail addresses instead of the real one?

Are all UTXOs locked by an address spent in a transaction?

Book about a time-travel war fought by computers

Deal the cards to the players

Would the melodic leap of the opening phrase of Mozart's K545 be considered dissonant?

Rsync delta copy not working for virtual disk

Remote file copy util (like rsync) but that will take account of data already copied (in this session)?rsync doesn't use delta transfer on first runHas anyone achieved true differential sync with rsync in ESXi?How to take periodic snapshots of VMs and efficiently transport them off-sitersync over ssh is not working anymore, while ssh itself is working fine (Write failed: broken pipe)rsync not deleting foldersGet Rsync status for outputVerify that rsync copied everything it was supposed to?re-sync a single large file using rsyncVirtual disk backup: rsync alternative?

0

I have tried rsync for copying VMs to remote host but the deltacopy does not seem to work. I am using Virtualbox on CentOS 6.2. My VM has 1 snapshot so I just copy the snapshot file instead of the entire base image. This works when I copy to the remote host mapped as a local drive (in which case rsync deltacopy is not used).

The problem is that when I use rsync over ssh to copy a simple text file changes are copied but when I use the same method to copy the snapshot file, changes (such as a text file created on the desktop) are not copied. Has anyone successfully used rsync to copy changes to a VM HDD without copying the whole file?

ssh virtual-machines rsync virtualbox centos6

share|improve this question

edited Aug 31 '12 at 7:36

Kenny Rasschaert

7,71733455

asked Aug 30 '12 at 12:27

ebabibictebabibict

12

bumped to the homepage by Community♦ 21 mins ago

This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.

add a comment | 

0

I have tried rsync for copying VMs to remote host but the deltacopy does not seem to work. I am using Virtualbox on CentOS 6.2. My VM has 1 snapshot so I just copy the snapshot file instead of the entire base image. This works when I copy to the remote host mapped as a local drive (in which case rsync deltacopy is not used).

The problem is that when I use rsync over ssh to copy a simple text file changes are copied but when I use the same method to copy the snapshot file, changes (such as a text file created on the desktop) are not copied. Has anyone successfully used rsync to copy changes to a VM HDD without copying the whole file?

ssh virtual-machines rsync virtualbox centos6

share|improve this question

edited Aug 31 '12 at 7:36

Kenny Rasschaert

7,71733455

asked Aug 30 '12 at 12:27

ebabibictebabibict

12

bumped to the homepage by Community♦ 21 mins ago

This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.

add a comment | 

I have tried rsync for copying VMs to remote host but the deltacopy does not seem to work. I am using Virtualbox on CentOS 6.2. My VM has 1 snapshot so I just copy the snapshot file instead of the entire base image. This works when I copy to the remote host mapped as a local drive (in which case rsync deltacopy is not used).

The problem is that when I use rsync over ssh to copy a simple text file changes are copied but when I use the same method to copy the snapshot file, changes (such as a text file created on the desktop) are not copied. Has anyone successfully used rsync to copy changes to a VM HDD without copying the whole file?

ssh virtual-machines rsync virtualbox centos6

share|improve this question

edited Aug 31 '12 at 7:36

Kenny Rasschaert

7,71733455

asked Aug 30 '12 at 12:27

ebabibictebabibict

12

share|improve this question

edited Aug 31 '12 at 7:36

Kenny Rasschaert

7,71733455

asked Aug 30 '12 at 12:27

ebabibictebabibict

12

share|improve this question

edited Aug 31 '12 at 7:36

Kenny Rasschaert

7,71733455

asked Aug 30 '12 at 12:27

ebabibictebabibict

12

edited Aug 31 '12 at 7:36

Kenny Rasschaert

7,71733455

edited Aug 31 '12 at 7:36

Kenny Rasschaert

7,71733455

asked Aug 30 '12 at 12:27

ebabibictebabibict

12

asked Aug 30 '12 at 12:27

ebabibictebabibict

12

1 Answer
1

active

oldest

votes

0

I think rsync probably does do its delta copy for a virtual disk, but the disk is so large that it needs to read the whole thing to find the differences. With multiple files (like a normal file system, not a virtual disk), rsync can use the individual timestamps and sizes of all your files, but with a virtual disk it just has the 1 file.

So if you are copying a file locally, it is reading the disk twice on the same machine (2 files, source and destination) making it very slow still, and if it's remote, the transfer over the wire is low, but the full disk is stll read on both sides by the 2 separate rsync processes during the rolling diff process. I haven't perfectly verified this, but watching iostat (with all disk activity not just my rsync test), I found the write speed to be very low compared to the read speed.

Also, when you copy with rsync, it creates a temporary file before overwriting the destination file. To avoid this, you can use --inplace. This way you never have a second copy written, but you still have the whole disk read, so it is not a perfect solution.

Originally I said: I don't think there is a way around reading the whole file... to do this you need something other than rsync, such as a copy-on-write file system like btrfs or zfs with incremental sending ability, which is the ability to already know the differences just from the file system metadata rather than reading all the data again.

Edit: Today I realized you could probably avoid reading the whole file if the file was split up in many parts, so each part has a different timestamp, etc. with the vmdk format's Split2G variant, which splits the file up into many 2GB files (but I haven't verified that the parts have different timestamps).

share|improve this answer

edited Nov 21 '12 at 14:18

answered Nov 20 '12 at 13:12

PeterPeter

1,88611121

add a comment | 

Your Answer

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "2"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f422858%2frsync-delta-copy-not-working-for-virtual-disk%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

0

I think rsync probably does do its delta copy for a virtual disk, but the disk is so large that it needs to read the whole thing to find the differences. With multiple files (like a normal file system, not a virtual disk), rsync can use the individual timestamps and sizes of all your files, but with a virtual disk it just has the 1 file.

So if you are copying a file locally, it is reading the disk twice on the same machine (2 files, source and destination) making it very slow still, and if it's remote, the transfer over the wire is low, but the full disk is stll read on both sides by the 2 separate rsync processes during the rolling diff process. I haven't perfectly verified this, but watching iostat (with all disk activity not just my rsync test), I found the write speed to be very low compared to the read speed.

Also, when you copy with rsync, it creates a temporary file before overwriting the destination file. To avoid this, you can use --inplace. This way you never have a second copy written, but you still have the whole disk read, so it is not a perfect solution.

Originally I said: I don't think there is a way around reading the whole file... to do this you need something other than rsync, such as a copy-on-write file system like btrfs or zfs with incremental sending ability, which is the ability to already know the differences just from the file system metadata rather than reading all the data again.

Edit: Today I realized you could probably avoid reading the whole file if the file was split up in many parts, so each part has a different timestamp, etc. with the vmdk format's Split2G variant, which splits the file up into many 2GB files (but I haven't verified that the parts have different timestamps).

share|improve this answer

edited Nov 21 '12 at 14:18

answered Nov 20 '12 at 13:12

PeterPeter

1,88611121

add a comment | 

0

I think rsync probably does do its delta copy for a virtual disk, but the disk is so large that it needs to read the whole thing to find the differences. With multiple files (like a normal file system, not a virtual disk), rsync can use the individual timestamps and sizes of all your files, but with a virtual disk it just has the 1 file.

So if you are copying a file locally, it is reading the disk twice on the same machine (2 files, source and destination) making it very slow still, and if it's remote, the transfer over the wire is low, but the full disk is stll read on both sides by the 2 separate rsync processes during the rolling diff process. I haven't perfectly verified this, but watching iostat (with all disk activity not just my rsync test), I found the write speed to be very low compared to the read speed.

Also, when you copy with rsync, it creates a temporary file before overwriting the destination file. To avoid this, you can use --inplace. This way you never have a second copy written, but you still have the whole disk read, so it is not a perfect solution.

Originally I said: I don't think there is a way around reading the whole file... to do this you need something other than rsync, such as a copy-on-write file system like btrfs or zfs with incremental sending ability, which is the ability to already know the differences just from the file system metadata rather than reading all the data again.

Edit: Today I realized you could probably avoid reading the whole file if the file was split up in many parts, so each part has a different timestamp, etc. with the vmdk format's Split2G variant, which splits the file up into many 2GB files (but I haven't verified that the parts have different timestamps).

share|improve this answer

edited Nov 21 '12 at 14:18

answered Nov 20 '12 at 13:12

PeterPeter

1,88611121

add a comment | 

I think rsync probably does do its delta copy for a virtual disk, but the disk is so large that it needs to read the whole thing to find the differences. With multiple files (like a normal file system, not a virtual disk), rsync can use the individual timestamps and sizes of all your files, but with a virtual disk it just has the 1 file.

So if you are copying a file locally, it is reading the disk twice on the same machine (2 files, source and destination) making it very slow still, and if it's remote, the transfer over the wire is low, but the full disk is stll read on both sides by the 2 separate rsync processes during the rolling diff process. I haven't perfectly verified this, but watching iostat (with all disk activity not just my rsync test), I found the write speed to be very low compared to the read speed.

Also, when you copy with rsync, it creates a temporary file before overwriting the destination file. To avoid this, you can use --inplace. This way you never have a second copy written, but you still have the whole disk read, so it is not a perfect solution.

Originally I said: I don't think there is a way around reading the whole file... to do this you need something other than rsync, such as a copy-on-write file system like btrfs or zfs with incremental sending ability, which is the ability to already know the differences just from the file system metadata rather than reading all the data again.

Edit: Today I realized you could probably avoid reading the whole file if the file was split up in many parts, so each part has a different timestamp, etc. with the vmdk format's Split2G variant, which splits the file up into many 2GB files (but I haven't verified that the parts have different timestamps).

share|improve this answer

edited Nov 21 '12 at 14:18

answered Nov 20 '12 at 13:12

PeterPeter

1,88611121

share|improve this answer

edited Nov 21 '12 at 14:18

answered Nov 20 '12 at 13:12

PeterPeter

1,88611121

answered Nov 20 '12 at 13:12

PeterPeter

1,88611121

answered Nov 20 '12 at 13:12

PeterPeter

1,88611121