How to Backup and Restore MBR within Logical Volumes?

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

How to Backup and Restore MBR within Logical Volumes?

"Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "
Hi All,

I have several Xen virtual machines within logical volumes using LVM2. I did not use disk images for performance reasons.

Conventionally, if I want to clone my virtual machines, I have to dd the LV to an image file. But this consumes a lot of time and harddisk space.

So, instead of doing that, I want to use losetup and kpartx with my logical volumes, which contain operating systems of virtual machines.

I can backup the filesystems of a virtual machine in this way:

# losetup /dev/loop1 /dev/virtualmachines/windows7-x64

virtualmachines is the name of my volume group.

windows7-x64 is the logical volume.

# kpartx -av /dev/loop1

Then I would see the partitions of the virtual machine within a logical volume, like so:

/dev/mapper/loop1p1
/dev/mapper/loop1p2
/dev/mapper/loop1p3

Now that I can access the partitions of the virtual machine within a logical volume, I can use partimage or fsarchiver to backup the partitions (provided the filesystem is supported by the archiver).

But the problem is that I can only backup/clone the filesystems of my virtual machine within a logical volume. I can't backup the Master Boot Record (MBR) of the virtual machine within a logical volume. For example,

dd if=/dev/hda of=mbr.hda bs=512 count=1

Because /dev/hda resides in a logical volume. The logical volume is a virtual harddisk for my virtual machine.

I would like to know how to backup and also restore the MBR of my virtual machine/guest operating system/domU within a logical volume because the losetup and kpartx procedure only allows me access to the partitions, not the MBR. A complete backup of a virtual machine (and also a bare metal machine) includes the MBR and all filesystems.

If there is a catastrophic failure with my logical volumes containing domUs, I would like to

1) re-create the physical volume (PV)

2) re-create the volume group

3) assign the PV to the volume group

4) restore the LVM metadata, i.e. the configuration files for all the logical volumes

5) restore the MBR of my domU

6) restore the filesystems of my domU

Please advise.

Thank you very much.

--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore

--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen
Reply | Threaded
Open this post in threaded view
|

Re: [Xen-users] How to Backup and Restore MBR within Logical Volumes?

"Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "
> dd if=/dev/virtualmachines/windows7-x64 of=mbr.w7-x64 bs=512 count=1

I think if you do this, you are only backing up the first 512 bytes of the logical volume, not the MBR.

Someone correct me if I am wrong.

--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore

On Thu, Nov 12, 2009 at 9:36 AM, Geert Janssens <[hidden email]> wrote:
On Thursday 12 November 2009, Mr. Teo En Ming (Zhang Enming) wrote:
> Hi All,
>
> I have several Xen virtual machines within logical volumes using LVM2. I
> did not use disk images for performance reasons.
>
> Conventionally, if I want to clone my virtual machines, I have to dd the LV
> to an image file. But this consumes a lot of time and harddisk space.
>
> So, instead of doing that, I want to use losetup and kpartx with my logical
> volumes, which contain operating systems of virtual machines.
>
> I can backup the filesystems of a virtual machine in this way:
>
> # losetup /dev/loop1 /dev/virtualmachines/windows7-x64
>
Are you sure you need to call losetup first ?

I remember I used kpartx directly on the lvm containing my vbd.

Also, I think you can treat the lvm based vbd as a real disk.

So
dd if=/dev/virtualmachines/windows7-x64 of=mbr.w7-x64 bs=512 count=1

should backup your mbr.

Anyone correct me if I'm wrong please.

Geert

--
Kobalt W.I.T.
Web & Information Technology
Brusselsesteenweg 152
1850 Grimbergen

Tel  : +32 479 339 655
Email: [hidden email]


_______________________________________________
Xen-users mailing list
[hidden email]
http://lists.xensource.com/xen-users





--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen
Reply | Threaded
Open this post in threaded view
|

Re: How to Backup and Restore MBR within Logical Volumes?

Veli-Pekka Kestilä
In reply to this post by "Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "
Mr. Teo En Ming (Zhang Enming) wrote:

> Hi All,
>
> I have several Xen virtual machines within logical volumes using LVM2.
> I did not use disk images for performance reasons.
>
> # losetup /dev/loop1 /dev/virtualmachines/windows7-x64
> windows7-x64 is the logical volume.
> # kpartx -av /dev/loop1
>
> Then I would see the partitions of the virtual machine within a
> logical volume, like so:
>
> /dev/mapper/loop1p1
> /dev/mapper/loop1p2
> /dev/mapper/loop1p3
>
> Now that I can access the partitions of the virtual machine within a
> logical volume, I can use partimage or fsarchiver to backup the
> partitions (provided the filesystem is supported by the archiver).
>
> But the problem is that I can only backup/clone the filesystems of my
> virtual machine within a logical volume. I can't backup the Master
> Boot Record (MBR) of the virtual machine within a logical volume. For
> example,
>
> dd if=/dev/hda of=mbr.hda bs=512 count=1
>
> Because /dev/hda resides in a logical volume. The logical volume is a
> virtual harddisk for my virtual machine.
You can do dd if=/dev/dev/loop1 of=mbr.dha bs=512 count=1 to make the
copy of the boot block.

>
> I would like to know how to backup and also restore the MBR of my
> virtual machine/guest operating system/domU within a logical volume
> because the losetup and kpartx procedure only allows me access to the
> partitions, not the MBR. A complete backup of a virtual machine (and
> also a bare metal machine) includes the MBR and all filesystems.
>
> If there is a catastrophic failure with my logical volumes containing
> domUs, I would like to
>
> 1) re-create the physical volume (PV)
>
> 2) re-create the volume group
>
> 3) assign the PV to the volume group
>
> 4) restore the LVM metadata, i.e. the configuration files for all the
> logical volumes
>
> 5) restore the MBR of my domU
>
> 6) restore the filesystems of my domU
>
Of these steps you can recreate 1,2,3 and 4 just by keeping records of
their parameters and using normal fdisk, pvcreate, vgcreate and lvcreate
commands.

For the things inside of lvm you can use losetup to make the loopback
device out of the lv where you want to install the quest operating
system and then recreate mbr from the file. Problem probably is how to
recreate the filesystems if your backup software cannot do it. Also with
windows protected system files can be a problem if backup software
doesn't support them.

One thing you could of course do is to make the system restore backup
inside of the DomU when it's running. And then boot new DomU when
starting up and use the system backup to restore. (of course this is
still more work that pure dd from one place to another.)

Problem is that everything else than using dd can have it fair share of
problems. If the domU can't be offline too long, you could allocate more
space and use the lvm:s instant cloning features (can't remember the
name just now. ) and then use dd with gzip or bzip to make the backup
from this clone to keep the space requirements minimum for the stored
copies. And have minimum disruption for the DomU.

-vpk

--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen
Reply | Threaded
Open this post in threaded view
|

Re: How to Backup and Restore MBR within Logical Volumes?

"Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "
In reply to this post by "Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "
> Maybe I didn't really understand your setup, but... isn't your mbr simply on /dev/loop1?

Yes, you are right.

root@sysresccd /mnt/hitachi/test % fdisk -l /dev/loop1

Disk /dev/loop1: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x2b362b35

      Device Boot      Start         End      Blocks   Id  System
/dev/loop1p1   *           1        6526    52420063+   7  HPFS/NTFS

I am so used with device nomenclature of harddisks being /dev/hda, /dev/hdb, /dev/sda, /dev/md0, /dev/xvda, etc that it didn't occur to me that the device node of my virtual machine in a logical volume is very simply /dev/loop1 after I have performed the "losetup /dev/loop1 /dev/volumegroup/logicalvolume" step.

Thank you very much for enlightening me! Now I can proceed with writing the backup and restore script for my Fedora 11 Dom0 (all partitions using LVM2 logical volumes) and all of my Xen virtual machines (contained in logical volumes).

--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore

On Thu, Nov 12, 2009 at 9:38 AM, Roberto Ragusa <[hidden email]> wrote:
Mr. Teo En Ming (Zhang Enming) wrote:

> # kpartx -av /dev/loop1
[...]
> But the problem is that I can only backup/clone the filesystems of my
> virtual machine within a logical volume. I can't backup the Master Boot
> Record (MBR) of the virtual machine within a logical volume. For example,
>
> dd if=/dev/hda of=mbr.hda bs=512 count=1
>
> Because /dev/hda resides in a logical volume. The logical volume is a
> virtual harddisk for my virtual machine.

Maybe I didn't really understand your setup, but... isn't your mbr
simply on /dev/loop1?

--
  Roberto Ragusa    mail at robertoragusa.it

--
fedora-list mailing list
[hidden email]
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
Guidelines: http://fedoraproject.org/wiki/Communicate/MailingListGuidelines





--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen
Reply | Threaded
Open this post in threaded view
|

Re: How to Backup and Restore MBR within Logical Volumes?

"Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "
In reply to this post by Veli-Pekka Kestilä
Thank you for your advice!

Is it called LVM snapshot?

--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore

2009/11/12 Veli-Pekka Kestilä <[hidden email]>
Mr. Teo En Ming (Zhang Enming) wrote:
Hi All,

I have several Xen virtual machines within logical volumes using LVM2. I did not use disk images for performance reasons.

# losetup /dev/loop1 /dev/virtualmachines/windows7-x64
windows7-x64 is the logical volume.
# kpartx -av /dev/loop1

Then I would see the partitions of the virtual machine within a logical volume, like so:

/dev/mapper/loop1p1
/dev/mapper/loop1p2
/dev/mapper/loop1p3

Now that I can access the partitions of the virtual machine within a logical volume, I can use partimage or fsarchiver to backup the partitions (provided the filesystem is supported by the archiver).

But the problem is that I can only backup/clone the filesystems of my virtual machine within a logical volume. I can't backup the Master Boot Record (MBR) of the virtual machine within a logical volume. For example,

dd if=/dev/hda of=mbr.hda bs=512 count=1

Because /dev/hda resides in a logical volume. The logical volume is a virtual harddisk for my virtual machine.
You can do dd if=/dev/dev/loop1 of=mbr.dha bs=512 count=1 to make the copy of the boot block.


I would like to know how to backup and also restore the MBR of my virtual machine/guest operating system/domU within a logical volume because the losetup and kpartx procedure only allows me access to the partitions, not the MBR. A complete backup of a virtual machine (and also a bare metal machine) includes the MBR and all filesystems.

If there is a catastrophic failure with my logical volumes containing domUs, I would like to

1) re-create the physical volume (PV)

2) re-create the volume group

3) assign the PV to the volume group

4) restore the LVM metadata, i.e. the configuration files for all the logical volumes

5) restore the MBR of my domU

6) restore the filesystems of my domU

Of these steps you can recreate 1,2,3 and 4 just by keeping records of their parameters and using normal fdisk, pvcreate, vgcreate and lvcreate commands.

For the things inside of lvm you can use losetup to make the loopback device out of the lv where you want to install the quest operating system and then recreate mbr from the file. Problem probably is how to recreate the filesystems if your backup software cannot do it. Also with windows protected system files can be a problem if backup software doesn't support them.

One thing you could of course do is to make the system restore backup inside of the DomU when it's running. And then boot new DomU when starting up and use the system backup to restore. (of course this is still more work that pure dd from one place to another.)

Problem is that everything else than using dd can have it fair share of problems. If the domU can't be offline too long, you could allocate more space and use the lvm:s instant cloning features (can't remember the name just now. ) and then use dd with gzip or bzip to make the backup from this clone to keep the space requirements minimum for the stored copies. And have minimum disruption for the DomU.

-vpk

--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen





--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen
Reply | Threaded
Open this post in threaded view
|

Re: How to Backup and Restore MBR within Logical Volumes?

Veli-Pekka Kestilä
Mr. Teo En Ming (Zhang Enming) wrote:
> Thank you for your advice!
>
> Is it called LVM snapshot?
Yes. That's what it's called for.

-vpk

PS. And please don't top post. It breaks the 'flow' of the post and
makes it hard to read from the archives. Plus is against the list rules
in some of the lists you are cross posting this (which is also bad). It
would have been better to ask the question separatedly on different
lists instead.

>
>
>
>     Problem is that everything else than using dd can have it fair
>     share of problems. If the domU can't be offline too long, you
>     could allocate more space and use the lvm:s instant cloning
>     features (can't remember the name just now. ) and then use dd with
>     gzip or bzip to make the backup from this clone to keep the space
>     requirements minimum for the stored copies. And have minimum
>     disruption for the DomU.
>
>     -vpk
>
>     --
>     Fedora-xen mailing list
>     [hidden email] <mailto:[hidden email]>
>     https://www.redhat.com/mailman/listinfo/fedora-xen
>
>
>
>
>

--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen
Reply | Threaded
Open this post in threaded view
|

Re: [Xen-users] Re: How to Backup and Restore MBR within Logical Volumes?

"Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "
In reply to this post by "Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "
Hi,

I don't think I will be using the LVM snapshot method. I will simply ensure that my domU is not running, then I proceed to do the losetup and kpartx procedure. Subsequently I will backup the MBR of my domU by dd-ing /dev/loopX and clone all the filesystems of the domU using fsarchiver.

I would like to use fsarchiver because it can support LZMA compression. LZMA can compress better than bzip2 and decompress 3x faster than bzip2. partimage only supports gzip and bzip2.

--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore

On Thu, Nov 12, 2009 at 10:02 AM, Fajar A. Nugraha <[hidden email]> wrote:
On Thu, Nov 12, 2009 at 4:52 PM, Mr. Teo En Ming (Zhang Enming)
<[hidden email]> wrote:
> Now I can proceed with writing the
> backup and restore script for my Fedora 11 Dom0 (all partitions using LVM2
> logical volumes) and all of my Xen virtual machines (contained in logical
> volumes).

A reminder though: don't forget to create a "consistent", unchanging
version of domU storage before doing backup. Either by shutting down
domU, or by creating LVM snapshot first.

Also, if it comes to Windows domUs, I find ntfsclone is better than
fsarchiver. One of the reasons is that fsarchiver can not create
compressed files during restore.

--
Fajar





--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen
Reply | Threaded
Open this post in threaded view
|

Re: How to Backup and Restore MBR within Logical Volumes?

"Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "
In reply to this post by Veli-Pekka Kestilä


2009/11/12 Veli-Pekka Kestilä <[hidden email]>
Mr. Teo En Ming (Zhang Enming) wrote:
Thank you for your advice!

Is it called LVM snapshot?
Yes. That's what it's called for.

-vpk

PS. And please don't top post. It breaks the 'flow' of the post and makes it hard to read from the archives. Plus is against the list rules in some of the lists you are cross posting this (which is also bad). It would have been better to ask the question separatedly on different lists instead.



   Problem is that everything else than using dd can have it fair
   share of problems. If the domU can't be offline too long, you
   could allocate more space and use the lvm:s instant cloning
   features (can't remember the name just now. ) and then use dd with
   gzip or bzip to make the backup from this clone to keep the space
   requirements minimum for the stored copies. And have minimum
   disruption for the DomU.

   -vpk

   --
   Fedora-xen mailing list
   [hidden email] <mailto:[hidden email]>


Thank you vpk.

I will keep that in mind when I have a new question to ask.

--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore

--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen
Reply | Threaded
Open this post in threaded view
|

Re: How to Backup and Restore MBR within Logical Volumes?

"Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "
In reply to this post by "Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "


On Thu, Nov 12, 2009 at 10:45 AM, Bryn M. Reeves <[hidden email]> wrote:
On Thu, 2009-11-12 at 09:23 +0000, Mr. Teo En Ming (Zhang Enming) wrote:
> Hi All,
>
> I have several Xen virtual machines within logical volumes using LVM2.
> I did not use disk images for performance reasons.
>
> Conventionally, if I want to clone my virtual machines, I have to dd
> the LV to an image file. But this consumes a lot of time and harddisk
> space.
>
> So, instead of doing that, I want to use losetup and kpartx with my
> logical volumes, which contain operating systems of virtual machines.
>
> I can backup the filesystems of a virtual machine in this way:
>
> # losetup /dev/loop1 /dev/virtualmachines/windows7-x64

What's the point of adding a loopback device on top of the LV? Running
kpartx on the LV itself will work just fine and this just adds an
unnecessary layer of overhead and complexity unless I am missing
something.

> dd if=/dev/hda of=mbr.hda bs=512 count=1
>
> Because /dev/hda resides in a logical volume. The logical volume is a
> virtual harddisk for my virtual machine.

Assuming that the LV given above is a whole-disk image containing a DOS
MBR partition table:

dd if=/dev/virtualmachines/windows7-x64 of=/tmp.mbr.img bs=512 count=1

You could also do the same with the loopN device that you set up
earlier, although I still don't see the need for that step.

> 1) re-create the physical volume (PV)
>
> 2) re-create the volume group
>
> 3) assign the PV to the volume group
>
> 4) restore the LVM metadata, i.e. the configuration files for all the
> logical volumes
>
> 5) restore the MBR of my domU
>
> 6) restore the filesystems of my domU

Should work fine, just be sure to test each step so that you are
confident and comfortable with it before you find yourself needing to do
this in anger.

Regards,
Bryn.


--

Hi Bryn,

I have just tried an experiment without the losetup step. I have verified that it works using kpartx only.

root@sysresccd /root % kpartx -av /dev/virtualmachines/winxphome32
add map virtualmachines-winxphome32p1 (253:22): 0 104840127 linear /dev/virtualmachines/winxphome32 63
root@sysresccd /root % cd /dev/mapper
root@sysresccd /dev/mapper % ls *winxphome32*
virtualmachines-winxphome32  virtualmachines-winxphome32p1
root@sysresccd /dev/mapper % ls -al *winxphome32*
brw-rw---- 1 root disk 253,  0 2009-11-12 15:49 virtualmachines-winxphome32
brw-rw---- 1 root disk 253, 22 2009-11-12 18:47 virtualmachines-winxphome32p1
root@sysresccd /dev/mapper % ls -al *winxphome32*
brw-rw---- 1 root disk 253,  0 2009-11-12 15:49 virtualmachines-winxphome32
brw-rw---- 1 root disk 253, 22 2009-11-12 18:47 virtualmachines-winxphome32p1
root@sysresccd /dev/mapper % fdisk -l virtualmachines-winxphome32

Disk virtualmachines-winxphome32: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x2b362b35

                       Device Boot      Start         End      Blocks   Id  System
virtualmachines-winxphome32p1   *           1        6526    52420063+   7  HPFS/NTFS

Without the losetup and the corresponding overhead, it will speed up cloning of my virtual machines within LVs.


--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore

--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen
Reply | Threaded
Open this post in threaded view
|

Re: How to Backup and Restore MBR within Logical Volumes?

"Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "


On Thu, Nov 12, 2009 at 10:55 AM, Mr. Teo En Ming (Zhang Enming) <[hidden email]> wrote:


On Thu, Nov 12, 2009 at 10:45 AM, Bryn M. Reeves <[hidden email]> wrote:
On Thu, 2009-11-12 at 09:23 +0000, Mr. Teo En Ming (Zhang Enming) wrote:
> Hi All,
>
> I have several Xen virtual machines within logical volumes using LVM2.
> I did not use disk images for performance reasons.
>
> Conventionally, if I want to clone my virtual machines, I have to dd
> the LV to an image file. But this consumes a lot of time and harddisk
> space.
>
> So, instead of doing that, I want to use losetup and kpartx with my
> logical volumes, which contain operating systems of virtual machines.
>
> I can backup the filesystems of a virtual machine in this way:
>
> # losetup /dev/loop1 /dev/virtualmachines/windows7-x64

What's the point of adding a loopback device on top of the LV? Running
kpartx on the LV itself will work just fine and this just adds an
unnecessary layer of overhead and complexity unless I am missing
something.

> dd if=/dev/hda of=mbr.hda bs=512 count=1
>
> Because /dev/hda resides in a logical volume. The logical volume is a
> virtual harddisk for my virtual machine.

Assuming that the LV given above is a whole-disk image containing a DOS
MBR partition table:

dd if=/dev/virtualmachines/windows7-x64 of=/tmp.mbr.img bs=512 count=1

You could also do the same with the loopN device that you set up
earlier, although I still don't see the need for that step.

> 1) re-create the physical volume (PV)
>
> 2) re-create the volume group
>
> 3) assign the PV to the volume group
>
> 4) restore the LVM metadata, i.e. the configuration files for all the
> logical volumes
>
> 5) restore the MBR of my domU
>
> 6) restore the filesystems of my domU

Should work fine, just be sure to test each step so that you are
confident and comfortable with it before you find yourself needing to do
this in anger.

Regards,
Bryn.


--

Hi Bryn,

I have just tried an experiment without the losetup step. I have verified that it works using kpartx only.

root@sysresccd /root % kpartx -av /dev/virtualmachines/winxphome32
add map virtualmachines-winxphome32p1 (253:22): 0 104840127 linear /dev/virtualmachines/winxphome32 63
root@sysresccd /root % cd /dev/mapper
root@sysresccd /dev/mapper % ls *winxphome32*
virtualmachines-winxphome32  virtualmachines-winxphome32p1
root@sysresccd /dev/mapper % ls -al *winxphome32*
brw-rw---- 1 root disk 253,  0 2009-11-12 15:49 virtualmachines-winxphome32
brw-rw---- 1 root disk 253, 22 2009-11-12 18:47 virtualmachines-winxphome32p1
root@sysresccd /dev/mapper % ls -al *winxphome32*
brw-rw---- 1 root disk 253,  0 2009-11-12 15:49 virtualmachines-winxphome32
brw-rw---- 1 root disk 253, 22 2009-11-12 18:47 virtualmachines-winxphome32p1
root@sysresccd /dev/mapper % fdisk -l virtualmachines-winxphome32

Disk virtualmachines-winxphome32: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x2b362b35


                       Device Boot      Start         End      Blocks   Id  System
virtualmachines-winxphome32p1   *           1        6526    52420063+   7  HPFS/NTFS

Without the losetup and the corresponding overhead, it will speed up cloning of my virtual machines within LVs.



--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore

Hi All,

I have performed a backup of my F11 Dom0's PV (60 GB) of 3 logical volumes using fsarchiver with the maximum possible LZMA compression. The operation took 4 hours with 2 compression threads. The resulting backup archive size is 8.1 GB for a PV of 60 GB.

My hardware specs:

Intel Pentium Dual Core E6300 2.8 GHz
Intel Desktop Board DQ45CB
6 GB DDR2-800 memory

--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore

--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen
Reply | Threaded
Open this post in threaded view
|

Re: How to Backup and Restore MBR within Logical Volumes?

"Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "


On Thu, Nov 12, 2009 at 12:32 PM, Mr. Teo En Ming (Zhang Enming) <[hidden email]> wrote:


On Thu, Nov 12, 2009 at 10:55 AM, Mr. Teo En Ming (Zhang Enming) <[hidden email]> wrote:


On Thu, Nov 12, 2009 at 10:45 AM, Bryn M. Reeves <[hidden email]> wrote:
On Thu, 2009-11-12 at 09:23 +0000, Mr. Teo En Ming (Zhang Enming) wrote:
> Hi All,
>
> I have several Xen virtual machines within logical volumes using LVM2.
> I did not use disk images for performance reasons.
>
> Conventionally, if I want to clone my virtual machines, I have to dd
> the LV to an image file. But this consumes a lot of time and harddisk
> space.
>
> So, instead of doing that, I want to use losetup and kpartx with my
> logical volumes, which contain operating systems of virtual machines.
>
> I can backup the filesystems of a virtual machine in this way:
>
> # losetup /dev/loop1 /dev/virtualmachines/windows7-x64

What's the point of adding a loopback device on top of the LV? Running
kpartx on the LV itself will work just fine and this just adds an
unnecessary layer of overhead and complexity unless I am missing
something.

> dd if=/dev/hda of=mbr.hda bs=512 count=1
>
> Because /dev/hda resides in a logical volume. The logical volume is a
> virtual harddisk for my virtual machine.

Assuming that the LV given above is a whole-disk image containing a DOS
MBR partition table:

dd if=/dev/virtualmachines/windows7-x64 of=/tmp.mbr.img bs=512 count=1

You could also do the same with the loopN device that you set up
earlier, although I still don't see the need for that step.

> 1) re-create the physical volume (PV)
>
> 2) re-create the volume group
>
> 3) assign the PV to the volume group
>
> 4) restore the LVM metadata, i.e. the configuration files for all the
> logical volumes
>
> 5) restore the MBR of my domU
>
> 6) restore the filesystems of my domU

Should work fine, just be sure to test each step so that you are
confident and comfortable with it before you find yourself needing to do
this in anger.

Regards,
Bryn.


--

Hi Bryn,

I have just tried an experiment without the losetup step. I have verified that it works using kpartx only.

root@sysresccd /root % kpartx -av /dev/virtualmachines/winxphome32
add map virtualmachines-winxphome32p1 (253:22): 0 104840127 linear /dev/virtualmachines/winxphome32 63
root@sysresccd /root % cd /dev/mapper
root@sysresccd /dev/mapper % ls *winxphome32*
virtualmachines-winxphome32  virtualmachines-winxphome32p1
root@sysresccd /dev/mapper % ls -al *winxphome32*
brw-rw---- 1 root disk 253,  0 2009-11-12 15:49 virtualmachines-winxphome32
brw-rw---- 1 root disk 253, 22 2009-11-12 18:47 virtualmachines-winxphome32p1
root@sysresccd /dev/mapper % ls -al *winxphome32*
brw-rw---- 1 root disk 253,  0 2009-11-12 15:49 virtualmachines-winxphome32
brw-rw---- 1 root disk 253, 22 2009-11-12 18:47 virtualmachines-winxphome32p1
root@sysresccd /dev/mapper % fdisk -l virtualmachines-winxphome32

Disk virtualmachines-winxphome32: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x2b362b35


                       Device Boot      Start         End      Blocks   Id  System
virtualmachines-winxphome32p1   *           1        6526    52420063+   7  HPFS/NTFS

Without the losetup and the corresponding overhead, it will speed up cloning of my virtual machines within LVs.



--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore

Hi All,

I have performed a backup of my F11 Dom0's PV (60 GB) of 3 logical volumes using fsarchiver with the maximum possible LZMA compression. The operation took 4 hours with 2 compression threads. The resulting backup archive size is 8.1 GB for a PV of 60 GB.

My hardware specs:

Intel Pentium Dual Core E6300 2.8 GHz
Intel Desktop Board DQ45CB
6 GB DDR2-800 memory


--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore

The fsarchiver command I have used is:

fsarchiver savefs -v -z 9 -j 2 /mnt/hitachi/test/test.fsa /dev/vg_fedora11_host/lv_home /dev/vg_fedora11_host/lv_root /dev/vg_fedora11_host/lv_var

--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore

--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen
Reply | Threaded
Open this post in threaded view
|

Re: [Xen-users] How to Backup and Restore MBR within Logical Volumes?

"Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "
In reply to this post by "Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "


On Thu, Nov 12, 2009 at 6:18 PM, Geert Janssens <[hidden email]> wrote:
On Thursday 12 November 2009, you wrote:
> > dd if=/dev/virtualmachines/windows7-x64 of=mbr.w7-x64 bs=512 count=1
>
> I think if you do this, you are only backing up the first 512 bytes of the
> logical volume, not the MBR.
>
> Someone correct me if I am wrong.

I did some tests just to be sure. As far as I can tell, dd interacts with lvm
in exactly the same way as with a physical disk or a loop device.

In the test I copied the first sector directly from the lvm partition or via
the loop device. It results in exactly the same sector being copied.

Also, if you try fdisk -l on the lvm disk or the loop device, it results in
the same output.

Below is the output from my tests:

[root@aragorn:~]# losetup /dev/loop1 /dev/base/kobaltwit_f11_disk

[root@aragorn:~]# fdisk -l /dev/base/kobaltwit_f11_disk

Disk /dev/base/kobaltwit_f11_disk: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

                      Device Boot      Start         End      Blocks   Id
System
/dev/base/kobaltwit_f11_disk1   *           1        1305    10482381   83
Linux
[root@aragorn:~]# fdisk -l /dev/base/kobaltwit_f11_disk

Disk /dev/base/kobaltwit_f11_disk: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

                      Device Boot      Start         End      Blocks   Id
System
/dev/base/kobaltwit_f11_disk1   *           1        1305    10482381   83
Linux

[root@aragorn:~]# dd if=/dev/base/kobaltwit_f11_disk of=mbr.lvm bs=512 count=1
1+0 records in
1+0 records out
512 bytes (512 B) copied, 7.6e-05 seconds, 6.7 MB/s

[root@aragorn:~]# dd if=/dev/loop1 of=mbr.loop bs=512 count=1
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.000151 seconds, 3.4 MB/s

[root@aragorn:~]# diff mbr.l*
[root@aragorn:~]#

These test seem to indicate to me that the lvm layer in completely transparent
to userland tools such as fdisk or dd.

So I still think the losetup step is superfluous and possibly causing
unnecessary overhead.

Geert

--
Kobalt W.I.T.
Web & Information Technology
Brusselsesteenweg 152
1850 Grimbergen

Tel  : +32 479 339 655
Email: [hidden email]


Thank you!

I will omit the losetup step.

--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore

--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen
Reply | Threaded
Open this post in threaded view
|

Re: [Xen-users] Re: How to Backup and Restore MBR within Logical Volumes?

"Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "
In reply to this post by "Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "


On Thu, Nov 12, 2009 at 6:37 PM, Fajar A. Nugraha <[hidden email]> wrote:
On Thu, Nov 12, 2009 at 5:10 PM, Mr. Teo En Ming (Zhang Enming)
<[hidden email]> wrote:
> Hi,
>
> I don't think I will be using the LVM snapshot method. I will simply ensure
> that my domU is not running,

Ah, so you can live with the down time. It makes things a lot simpler then.

> then I proceed to do the losetup and kpartx
> procedure.

Like others mentioned, you don't need losetup for LVs. It will only
make things slower.

> Subsequently I will backup the MBR of my domU by dd-ing
> /dev/loopX and clone all the filesystems of the domU using fsarchiver.
>
> I would like to use fsarchiver because it can support LZMA compression. LZMA
> can compress better than bzip2 and decompress 3x faster than bzip2.
> partimage only supports gzip and bzip2.


If your concern is the size and speed of backup, here's one final note
from me: you might want to look at zfs. Possible use scenarios :
- use opensolaris as dom0. Probably the most supported option when it
comes to xen + zfs, but you need to be familiar with how opensolaris
works.
- use external storage server running solaris/opensolaris as iscsi
SAN. Think of it as poor-man's Netapp.
- use zfs-fuse inside Linux dom0, and store domU storage as files
- use zfs-fuse on domU, and do backup from domU instead of dom0.

With zfs, you'd get zfs snapshot (which is like instaneous in-place
backup), incremental block-level send (can greatly reduce backup
storage size if your data doesn't change often), optional transparent
compression for live data (like what ntfs has).

--
Fajar

I am not familiar with OpenSolaris but that may change with time.

--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore


--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen
Reply | Threaded
Open this post in threaded view
|

Re: [Xen-users] Re: How to Backup and Restore MBR within Logical Volumes?

"Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "


On Thu, Nov 12, 2009 at 8:53 PM, Mr. Teo En Ming (Zhang Enming) <[hidden email]> wrote:


On Thu, Nov 12, 2009 at 6:37 PM, Fajar A. Nugraha <[hidden email]> wrote:
On Thu, Nov 12, 2009 at 5:10 PM, Mr. Teo En Ming (Zhang Enming)
<[hidden email]> wrote:
> Hi,
>
> I don't think I will be using the LVM snapshot method. I will simply ensure
> that my domU is not running,

Ah, so you can live with the down time. It makes things a lot simpler then.

> then I proceed to do the losetup and kpartx
> procedure.

Like others mentioned, you don't need losetup for LVs. It will only
make things slower.

> Subsequently I will backup the MBR of my domU by dd-ing
> /dev/loopX and clone all the filesystems of the domU using fsarchiver.
>
> I would like to use fsarchiver because it can support LZMA compression. LZMA
> can compress better than bzip2 and decompress 3x faster than bzip2.
> partimage only supports gzip and bzip2.


If your concern is the size and speed of backup, here's one final note
from me: you might want to look at zfs. Possible use scenarios :
- use opensolaris as dom0. Probably the most supported option when it
comes to xen + zfs, but you need to be familiar with how opensolaris
works.
- use external storage server running solaris/opensolaris as iscsi
SAN. Think of it as poor-man's Netapp.
- use zfs-fuse inside Linux dom0, and store domU storage as files
- use zfs-fuse on domU, and do backup from domU instead of dom0.

With zfs, you'd get zfs snapshot (which is like instaneous in-place
backup), incremental block-level send (can greatly reduce backup
storage size if your data doesn't change often), optional transparent
compression for live data (like what ntfs has).

--
Fajar

I am not familiar with OpenSolaris but that may change with time.


--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore


Hi,

I have just finished writing my backup/cloning script. Could anyone help me verify whether my backup script will work?

<SCRIPT>

#!/bin/sh

# Script to Backup/Clone Xen Host/Dom0 and all DomUs which are using Logical Volumes as Virtual Hard Disks

# Written by:
 
# Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
# Alma Maters:
# (1) Singapore Polytechnic
# (2) National University of Singapore
# Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
# Secondary Blog: http://enmingteo.wordpress.com
# Youtube Videos: http://www.youtube.com/user/enmingteo
# Email: [hidden email]
# MSN: [hidden email]
# Mobile Phone (Starhub Prepaid): +65-8369-2618
# Street: Bedok Reservoir Road
# Country: Singapore

# First written: 13 November 2009 Friday 12:38 A.M. Singapore time
# Last updated: 13 November 2009 Friday 12:38 A.M. Singapore time

# REFERNCE: "Geek Sheet: Bare-metal backup and recovery", May 7th, 2008, Jason Perlow
# URL: http://blogs.zdnet.com/BTL/?p=8759

# Usage Instructions:

# Boot up your computer/server with System Rescue CD version 1.3.2 for i386/amd64.
# Then execute this backup script. You need to adapt this script to work for your environment.
# Download System Rescue CD from http://www.sysresccd.org/Main_Page

###############################################################################################################
# Declare Variables
###############################################################################################################

HARDDISK=/dev/sda
DEST=/media/hitachi/test

###############################################################################################################
# Cloning Xen Host/Dom0
###############################################################################################################

# Backup MBR

dd if=$HARDDISK of=$DEST/f11-xen-dom0-sda.mbr bs=512 count=1

# Activate all logical volumes in all volume groups

vgchange -ay

# Backup UUIDs of PVs

pvdisplay > $DEST/f11-xen-dom0-pvdisplay.txt

# Backup LVM Metadata

# Backup the configuration of dedicated volume group for dom0

vgcfgbackup -d -v vg_fedora11_host -f $DEST/vg_fedora11_host.vgcfg.backup

# Backup the configuration of dedicated volume group for domUs

vgcfgbackup -d -v virtualmachines -f $DEST/virtualmachines.vgcfg.backup

# Backup /boot partition and all logical volumes of dedicated volume group for host/dom0

fsarchiver savefs -v -z 9 -j 2 $DEST/f11-xen-dom0-filesystems.fsa /dev/sda1 /dev/vg_fedora11_host/lv_home \
/dev/vg_fedora11_host/lv_root /dev/vg_fedora11_host/lv_var

###############################################################################################################
# Cloning Xen-based Virtual Machines/DomUs/Guest Operating Systems/VMs
###############################################################################################################

# The following 8 virtual machines have physical volumes, so it is not possible to image using partimage or
# fsarchiver directly. The only convenient way is to dd the logical volume for the VM, ie. sector by sector
# copy. Other methods would be very tedious.

# VM 1: Fedora 11 x86_64 PV domU

dd if=/dev/virtualmachines/f11-pv-hpc-node0001 | gzip --fast > $DEST/f11-pv-hpc-node0001.img

# VM 2: Fedora 11 x86_64 PV domU

dd if=/dev/virtualmachines/f11-pv-hpc-node0002 | gzip --fast > $DEST/f11-pv-hpc-node0002.img

# VM 3: Fedora 11 x86_64 PV domU

dd if=/dev/virtualmachines/f11-pv-hpc-node0003 | gzip --fast > $DEST/f11-pv-hpc-node0003.img

# VM 4: Fedora 11 x86_64 PV domU

dd if=/dev/virtualmachines/f11-pv-hpc-node0004 | gzip --fast > $DEST/f11-pv-hpc-node0004.img

# VM 5: Fedora 11 x86_64 PV domU

dd if=/dev/virtualmachines/f11-pv-hpc-node0005 | gzip --fast > $DEST/f11-pv-hpc-node0005.img

# VM 6: Fedora 11 x86_64 PV domU

dd if=/dev/virtualmachines/f11-pv-hpc-node0006 | gzip --fast > $DEST/f11-pv-hpc-node0006.img

# VM 7: Fedora 11 x86_64 PV domU

dd if=/dev/virtualmachines/f11-pv-hpc-node0007 | gzip --fast > $DEST/f11-pv-hpc-node0007.img

# VM 8: Fedora 11 x86_64 PV domU

dd if=/dev/virtualmachines/f11-pv-hpc-node0008 | gzip --fast > $DEST/f11-pv-hpc-node0008.img

# VM 9: FreeBSD 8.0 RC2 UNIX amd64 HVM domU

dd if=/dev/virtualmachines/freebsd | gzip --fast > $DEST/freebsd.img

# VM 10: OpenSolaris 2009.06 UNIX amd64 PV domU

dd if=/dev/virtualmachines/opensolaris | gzip --fast > $DEST/opensolaris.img

# VM 11: Rocks 5.1 x86_64 HPC Compute Cluster HVM domU

kpartx -av /dev/virtualmachines/rocks0001

dd if=/dev/virtualmachines/rocks0001 of=$DEST/rocks0001.mbr bs=512 count=1

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0001p1 $DEST/virtualmachines-rocks0001p1.img

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0001p2 $DEST/virtualmachines-rocks0001p2.img

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0001p5 $DEST/virtualmachines-rocks0001p5.img

kpartx -dv /dev/virtualmachines/rocks0001

# VM 12: Rocks 5.1 x86_64 HPC Compute Cluster HVM domU

kpartx -av /dev/virtualmachines/rocks0002

dd if=/dev/virtualmachines/rocks0002 of=$DEST/rocks0002.mbr bs=512 count=1

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0002p1 $DEST/virtualmachines-rocks0002p1.img

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0002p2 $DEST/virtualmachines-rocks0002p2.img

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0002p5 $DEST/virtualmachines-rocks0002p5.img

kpartx -dv /dev/virtualmachines/rocks0002

# VM 13: Rocks 5.1 x86_64 HPC Compute Cluster HVM domU

kpartx -av /dev/virtualmachines/rocks0003

dd if=/dev/virtualmachines/rocks0003 of=$DEST/rocks0003.mbr bs=512 count=1

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0003p1 $DEST/virtualmachines-rocks0003p1.img

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0003p2 $DEST/virtualmachines-rocks0003p2.img

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0003p5 $DEST/virtualmachines-rocks0003p5.img

kpartx -dv /dev/virtualmachines/rocks0003

# VM 14: Rocks 5.1 x86_64 HPC Compute Cluster HVM domU

kpartx -av /dev/virtualmachines/rocks0004

dd if=/dev/virtualmachines/rocks0004 of=$DEST/rocks0004.mbr bs=512 count=1

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0004p1 $DEST/virtualmachines-rocks0004p1.img

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0004p2 $DEST/virtualmachines-rocks0004p2.img

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0004p5 $DEST/virtualmachines-rocks0004p5.img

kpartx -dv /dev/virtualmachines/rocks0004

# VM 15: Rocks 5.1 x86_64 HPC Compute Cluster HVM domU

kpartx -av /dev/virtualmachines/rocks0005

dd if=/dev/virtualmachines/rocks0005 of=$DEST/rocks0005.mbr bs=512 count=1

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0005p1 $DEST/virtualmachines-rocks0005p1.img

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0005p2 $DEST/virtualmachines-rocks0005p2.img

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0005p5 $DEST/virtualmachines-rocks0005p5.img

kpartx -dv /dev/virtualmachines/rocks0005

# VM 16: Slackware64 13.0 amd64 HVM domU

kpartx -av /dev/virtualmachines/slackware64

dd if=/dev/virtualmachines/slackware64 of=$DEST/slackware64.mbr bs=512 count=1

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-slackware64p1 $DEST/virtualmachines-slackware64p1.img

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-slackware64p2 $DEST/virtualmachines-slackware64p2.img

kpartx -dv /dev/virtualmachines/slackware64

# VM 17: Ubuntu 9.10 Karmic Koala Linux HVM domU

kpartx -av /dev/virtualmachines/ubuntu910

dd if=/dev/virtualmachines/ubuntu910 of=$DEST/ubuntu910.mbr bs=512 count=1

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-ubuntu910p1 $DEST/virtualmachines-ubuntu910p1.img

kpartx -dv /dev/virtualmachines/ubuntu910

# VM 18: Windows XP Home Edition SP3 32-bit HVM domU with VGA passthrough (requires Intel VT-d)

kpartx -av /dev/virtualmachines/winxphome32

dd if=/dev/virtualmachines/winxphome32 of=$DEST/winxphome32.mbr bs=512 count=1

partimage -d -M -b -z1 save /dev/mapper/virtualmachines-winxphome32p1 $DEST/virtualmachines-winxphome32p1.img

kpartx -dv /dev/virtualmachines/winxphome32

# EOF

</SCRIPT>

Thank you!

--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore

--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen
Reply | Threaded
Open this post in threaded view
|

Re: [Xen-users] Re: How to Backup and Restore MBR within Logical Volumes?

"Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "


On Fri, Nov 13, 2009 at 5:40 AM, Sergey Vlasov <[hidden email]> wrote:
On Fri, Nov 13, 2009 at 12:57:22AM +0800, Mr. Teo En Ming (Zhang Enming) wrote:
[...]
> # VM 11: Rocks 5.1 x86_64 HPC Compute Cluster HVM domU
>
> kpartx -av /dev/virtualmachines/rocks0001
>
> dd if=/dev/virtualmachines/rocks0001 of=$DEST/rocks0001.mbr bs=512 count=1
>
> partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0001p1
> $DEST/virtualmachines-rocks0001p1.img
>
> partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0001p2
> $DEST/virtualmachines-rocks0001p2.img
>
> partimage -d -M -b -z1 save /dev/mapper/virtualmachines-rocks0001p5
> $DEST/virtualmachines-rocks0001p5.img

Note that this VM (and some other VMs listed in your script) uses
logical partitions.  In this case just saving a copy of MBR will not
be enough to save partition layout - MBR describes only 4 primary
partitions, and restoring just MBR will not restore extended
partitions.

One way to backup the complete partition layout is by saving also
the output of "sfdisk -d $device"; the resulting file can be used as
input to sfdisk to restore all partitions, including logical ones.
Saving MBR is still needed together with sfdisk, because it saves
the boot code (used for HVM) and CHS geometry information (which can
be used during boot in some cases).

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAkr8gM8ACgkQW82GfkQfsqLa+QCaAqpO5NWhYHtKVi3M5ytERw27
eC0AnjsJuG34MAR1jZRejBiJCVybQvc1
=RRGH
-----END PGP SIGNATURE-----

_______________________________________________
Xen-users mailing list
[hidden email]
http://lists.xensource.com/xen-users



Thank you for pointing this out. Otherwise I would made incomplete backups.

Would it be best practice to always backup both the MBR and the partition geometry using sfdisk whenever cloning our harddisks on desktops and servers?

Besides sfdisk, there are also other partitioning tools like fdisk, cfdisk, and parted.

Could these other tools also be used for backing up the partition geometry like sfdisk?

Thank you.

--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore

--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen
Reply | Threaded
Open this post in threaded view
|

RE: Re: [Xen-users] How to Backup and Restore MBR within Logical Volumes?

Dustin Henning
In reply to this post by "Singapore Citizen Mr. Teo En Ming (Zhang Enming) 张恩鸣 "
        As you are running Windows, I will assume that you are using HVM,
and therefore that /dev/virtualmachines/windows7-x64 is a hard disk image.
As such, the command you listed will back up the MBR, including the
partition table.  That isn't to say that Windows 7 doesn't have additional
boot information outside of the MBR, but that information would be included
in a file system backup.  Wikipedia has the structure of the MBR in a table
at http://en.wikipedia.org/wiki/Master_boot_record.  On that note, I often
use bs=446 when I want the MBR without the partition table, but if you are
backing up the partition as opposed to the file structure, then you might as
well keep the partition table (or even just back up the image).
        Dustin

From: [hidden email] [mailto:[hidden email]]
On Behalf Of Mr. Teo En Ming (Zhang Enming)
Sent: Thursday, November 12, 2009 04:45
To: Geert Janssens
Cc: Community assistance, encouragement, and advice for using Fedora.;
[hidden email]; Fedora Xen
Subject: [Fedora-xen] Re: [Xen-users] How to Backup and Restore MBR within
Logical Volumes?

> dd if=/dev/virtualmachines/windows7-x64 of=mbr.w7-x64 bs=512 count=1

I think if you do this, you are only backing up the first 512 bytes of the
logical volume, not the MBR.

Someone correct me if I am wrong.

--
Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) BEng(Hons)(Mechanical
Engineering)
Alma Maters:
(1) Singapore Polytechnic
(2) National University of Singapore
My Primary Blog: http://teo-en-ming-aka-zhang-enming.blogspot.com
My Secondary Blog: http://enmingteo.wordpress.com
My Youtube videos: http://www.youtube.com/user/enmingteo
Email: [hidden email]
Mobile Phone (Starhub Prepaid): +65-8369-2618
Street: Bedok Reservoir Road
Country: Singapore
On Thu, Nov 12, 2009 at 9:36 AM, Geert Janssens <[hidden email]> wrote:
On Thursday 12 November 2009, Mr. Teo En Ming (Zhang Enming) wrote:
> Hi All,
>
> I have several Xen virtual machines within logical volumes using LVM2. I
> did not use disk images for performance reasons.
>
> Conventionally, if I want to clone my virtual machines, I have to dd the
LV
> to an image file. But this consumes a lot of time and harddisk space.
>
> So, instead of doing that, I want to use losetup and kpartx with my
logical
> volumes, which contain operating systems of virtual machines.
>
> I can backup the filesystems of a virtual machine in this way:
>
> # losetup /dev/loop1 /dev/virtualmachines/windows7-x64
>
Are you sure you need to call losetup first ?

I remember I used kpartx directly on the lvm containing my vbd.

Also, I think you can treat the lvm based vbd as a real disk.

So
dd if=/dev/virtualmachines/windows7-x64 of=mbr.w7-x64 bs=512 count=1

should backup your mbr.

Anyone correct me if I'm wrong please.

Geert

--
Kobalt W.I.T.
Web & Information Technology
Brusselsesteenweg 152
1850 Grimbergen

Tel  : +32 479 339 655
Email: [hidden email]


_______________________________________________
Xen-users mailing list
[hidden email]
http://lists.xensource.com/xen-users






--
Fedora-xen mailing list
[hidden email]
https://www.redhat.com/mailman/listinfo/fedora-xen