Cloud Migration Guide: Difference between revisions

From bwCloud-OS
Jump to navigation Jump to search
No edit summary
 
(8 intermediate revisions by the same user not shown)
Line 11: Line 11:
{| class="mw-message-box mw-message-box-warning"
{| class="mw-message-box mw-message-box-warning"
| style="vertical-align:middle;" | '''⚠️ Entitlement and Quota Notice:'''  
| style="vertical-align:middle;" | '''⚠️ Entitlement and Quota Notice:'''  
For bwCloud-OS Gen3, entitlements and quotas are assigned exclusively by your '''home institution'''. A valid bwCloud-OS Gen3 entitlement is required to register for the service. During registration, your initial project quota is determined by the value assigned to your account by your home institution. This quota may be zero or a default value that does not meet the requirements of your intended project.
For bwCloud-OS Gen3, entitlements and quotas are assigned exclusively by your '''home institution'''. A valid bwCloud-OS Gen3 [[Registration#Entitlement_and_Eligibility|entitlement]] is required to register for the service. During registration, your initial project quota is determined by the value assigned to your account by your home institution. This quota may be zero or a default value that does not meet the requirements of your intended project.


To avoid duplication of effort, please inform yourself about your institution's current policy for assigning quotas before registering for the service. The appropriate contact point for such matters is your institution's '''local IT support'''. The bwCloud-OS team (via the bwSupportPortal) cannot grant or modify entitlements or quotas and cannot provide information about your home institution's policies.
To avoid duplication of effort, please inform yourself about your institution's current policy for assigning quotas before registering for the service. The appropriate contact point for such matters is your institution's '''local IT support'''. The bwCloud-OS team (via the bwSupportPortal) cannot grant or modify entitlements or quotas and cannot provide information about your home institution's policies.
Line 18: Line 18:


{{InANutshell|
{{InANutshell|
<li>Migration requires a new entitlement for Gen3 and appropriate quotas.</li>
<li>Migration requires a new [[Registration#Entitlement_and_Eligibility|'''entitlement''']] for Gen3 and appropriate quotas.</li>
<li>There is no automatic migration path: Gen2 and Gen3 are completely separate environments.</li>
<li>There is no automatic migration path: Gen2 and Gen3 are completely separate environments.</li>
<li>Network-related configuration (IP addresses, DNS records and resolvers, security groups, etc.) will change, and manual reconfiguration may be needed.</li>
<li>Network-related configuration (IP addresses, DNS records and resolvers, security groups, etc.) will change, and manual reconfiguration may be needed.</li>
Line 36: Line 36:


* '''Missing Entitlement?''' If you have not yet been assigned an entitlement, please get in touch directly with your local institution or university's IT support.  
* '''Missing Entitlement?''' If you have not yet been assigned an entitlement, please get in touch directly with your local institution or university's IT support.  
* '''Quota Allocation:''' Please note that the allocation of resource quotas is entirely managed and regulated through your respective institution.
* '''Quota Allocation:''' Please note that the allocation of resource quotas is entirely regulated through your respective institution.


Details can be found [https://wiki.bwcloud-os.de/index.php/Guide:_Entitlement_%26_Eligibility here].
Details can be found [https://wiki.bwcloud-os.de/index.php/Guide:_Entitlement_%26_Eligibility here].
Line 45: Line 45:


== Dashboard Login ==
== Dashboard Login ==
* [https://portal.bw-cloud.org/ Old bwCloud-OS (Gen2)]
* [{{DashboardURL(Gen2)}} Old bwCloud-OS (Gen2)]
* [https://portal.ul.bwcos.de/auth/login/?next=/%20https://portal.ul.bwcos.de/ New bwCloud-OS (Gen3)]
* [{{DashboardURL}} New bwCloud-OS (Gen3)]
 
== Networks ==
== Networks ==
The new infrastructure includes not only new networks but also new network-related features such as load balancers. You can find an overview [[Networks_Gen3|here]].
The new infrastructure includes not only new networks but also new network-related features such as load balancers. You can find an overview [[Networks|here]].


Network-specific differences between regions and details about the individual networks can be found in the following guide:
Network-specific differences between regions and details about the individual networks can be found in the following guide:
Line 65: Line 66:


'''Step-by-Step Guide:'''
'''Step-by-Step Guide:'''
* '''Prepare the Destination:''' Start a fresh VM in the Gen3 environment. If you need a separate volume for your data, please follow the [https://wiki.bwcloud-os.de/index.php/Guide:_Volumes_and_Images Guide: Volumes and Images] to create, attach, and mount it.
* '''Prepare the destination:''' Start a fresh VM in the Gen3 environment. If you need a separate volume for your data, please follow the [https://wiki.bwcloud-os.de/index.php/Guide:_Volumes_and_Images Guide: Volumes and Images] to create, attach, and mount it.
* '''Establish Connectivity:''' Ensure that your new Gen3 VM is reachable from your old Gen2 VM via SSH. You can achieve this by adding an SSH public key from the old VM to the <code>authorized_keys</code> file on the new VM, or by temporarily enabling password authentication. Verify the SSH connectivity before starting the transfer below.
* '''Establish connectivity:''' Ensure that your new Gen3 VM is reachable from your old Gen2 VM via SSH. You can achieve this by adding an SSH public key from the old VM to the <code>authorized_keys</code> file on the new VM, or by temporarily enabling password authentication. Verify the SSH connectivity before starting the transfer below.
* '''Sync Your Data:''' Use '''rsync''' to securely copy your files and directories directly over the network:
* '''Sync your data:''' Use '''rsync''' to securely copy your files and directories directly over the network:


<pre>
<pre>
Line 125: Line 126:
|}
|}
{| class="mw-message-box mw-message-box-notice"
{| class="mw-message-box mw-message-box-notice"
| style="vertical-align:middle;" | ⚠️ While it is technically possible to upload the downloaded <code>.img</code> file via the '''Horizon Dashboard''' (by navigating to '''Compute -> Images'''), this method is highly prone to errors, especially with large image files. We strongly advise avoiding the dashboard for this task and using the CLI method described above instead to prevent corrupted or interrupted uploads.
| style="vertical-align:middle;" | ⚠️ While it is technically possible to upload the downloaded image file via the '''Horizon Dashboard''' (by navigating to '''Compute -> Images'''), this method is highly prone to errors, especially with large image files. We strongly advise avoiding the dashboard for this task and using the CLI method described above instead to prevent corrupted or interrupted uploads.
|}
|}


Line 135: Line 136:
* Navigate to '''Project -> Compute -> Images'''. The uploaded snapshot should be listed there.
* Navigate to '''Project -> Compute -> Images'''. The uploaded snapshot should be listed there.
* Click on "Launch" (or "Start"), choose an instance name, and select the appropriate flavor, network, and SSH key.
* Click on "Launch" (or "Start"), choose an instance name, and select the appropriate flavor, network, and SSH key.


{| class="mw-message-box mw-message-box-notice"
{| class="mw-message-box mw-message-box-notice"
Line 142: Line 144:
* '''SSH Keys:''' Similar to the username, the new Gen3 environment might not successfully inject newly selected SSH keys into a migrated image. If you cannot log in with the newly assigned key, you will need to use your old Gen2 private SSH key to access the VM.
* '''SSH Keys:''' Similar to the username, the new Gen3 environment might not successfully inject newly selected SSH keys into a migrated image. If you cannot log in with the newly assigned key, you will need to use your old Gen2 private SSH key to access the VM.
* '''Network Configuration & Security:''' You will receive a '''new IP address and FQDN''', and Security Groups do not migrate. Therefore, you must manually recreate your (non-default) Security Groups in the Gen3 dashboard. Additionally, depending on the specific software running on your VM, you ''might'' need to update internal, network-dependent settings. If applicable, remember to adjust internal firewall rules, update Let's Encrypt certificates, or modify static IP configurations within your applications.
* '''Network Configuration & Security:''' You will receive a '''new IP address and FQDN''', and Security Groups do not migrate. Therefore, you must manually recreate your (non-default) Security Groups in the Gen3 dashboard. Additionally, depending on the specific software running on your VM, you ''might'' need to update internal, network-dependent settings. If applicable, remember to adjust internal firewall rules, update Let's Encrypt certificates, or modify static IP configurations within your applications.
* '''DNS Resolution:''' If you experience issues with domain name resolution within your VM after the migration, check your VM's DNS configuration (e.g., /etc/resolv.conf on many Linux systems). You may need to remove old Gen2 DNS server entries to restore functionality.
* '''DNS Resolution:''' If you experience issues with domain name resolution within your VM after the migration, check your VM's DNS configuration (e.g., <code>/etc/resolv.conf</code> on many Linux systems). You may need to remove old Gen2 DNS server entries to restore functionality.
|}
|}

Latest revision as of 14:38, 14 June 2026


⚠️ Important Shutdown Notice:

The bwCloud-OS Gen2 infrastructure will be permanently shut down in September 2026 (refer to the bwCloud-OS website for the timetable). All projects and data that have not been migrated to Gen3 by this date will be irrevocably deleted.

To continue using bwCloud-OS after the shutdown of Gen2, please ensure that you have obtained the necessary Gen3 entitlement and appropriate quotas (which are subject to costs) from your home institution in a timely manner.

⚠️ Entitlement and Quota Notice:

For bwCloud-OS Gen3, entitlements and quotas are assigned exclusively by your home institution. A valid bwCloud-OS Gen3 entitlement is required to register for the service. During registration, your initial project quota is determined by the value assigned to your account by your home institution. This quota may be zero or a default value that does not meet the requirements of your intended project.

To avoid duplication of effort, please inform yourself about your institution's current policy for assigning quotas before registering for the service. The appropriate contact point for such matters is your institution's local IT support. The bwCloud-OS team (via the bwSupportPortal) cannot grant or modify entitlements or quotas and cannot provide information about your home institution's policies.

In a Nutshell
  • Migration requires a new entitlement for Gen3 and appropriate quotas.
  • There is no automatic migration path: Gen2 and Gen3 are completely separate environments.
  • Network-related configuration (IP addresses, DNS records and resolvers, security groups, etc.) will change, and manual reconfiguration may be needed.
  • Recommended approach: Start a new, fresh instance (VM) in Gen3 and copy your old data over the network (e.g., via rsync).


General Information

This guide explains how to migrate your resources from the old bwCloud-OS (Gen2) environment to bwCloud-OS (Gen3). Because the two environments are completely separated, there is no automatic migration path. The recommended approach is to set up fresh instances in Gen3 and copy your data directly over the network using tools like rsync. Alternatively, you can manually export your resources (such as VM snapshots) from the old cloud, download them locally, and upload them to the new environment. Please note that network settings and Security Groups cannot be copied and must be manually reconfigured in the new environment.


Important Prerequisites: Entitlements and Quotas

The new bwCloud-OS (Gen3) will become a paid infrastructure platform. Therefore, it is necessary for each participating university or institution to actively assign the corresponding entitlements to its users. You cannot simply transfer your old projects directly without this authorization.

  • Missing Entitlement? If you have not yet been assigned an entitlement, please get in touch directly with your local institution or university's IT support.
  • Quota Allocation: Please note that the allocation of resource quotas is entirely regulated through your respective institution.

Details can be found here.

ℹ️ Note on Group Projects: The migration of complex group projects will take place in a second phase. The timetable is available here.

Dashboard Login

Networks

The new infrastructure includes not only new networks but also new network-related features such as load balancers. You can find an overview here.

Network-specific differences between regions and details about the individual networks can be found in the following guide:

➡️ Guide: Network Configuration by Region

Transferring Data and VMs

There are basically two ways to migrate to the new bwCloud-OS (Gen3): The recommended data transfer method via rsync over the network, and the snapshot method for transferring entire VMs.

Transferring Data (recommended)

Your data might reside directly on the root disk of your instance or on a separately attached Cinder volume. While it is technically possible to migrate an entire Cinder volume by converting it to an image and uploading it to the new cloud, we strongly advise against this approach. The process is extremely slow, and depending on the backend configuration, the upload may silently fail.

Our recommendation: Transfer your data directly over the network using rsync. This method works efficiently and universally, regardless of where your data is stored on the VM, and takes full advantage of the high network bandwidth available between the two cloud environments.

Step-by-Step Guide:

  • Prepare the destination: Start a fresh VM in the Gen3 environment. If you need a separate volume for your data, please follow the Guide: Volumes and Images to create, attach, and mount it.
  • Establish connectivity: Ensure that your new Gen3 VM is reachable from your old Gen2 VM via SSH. You can achieve this by adding an SSH public key from the old VM to the authorized_keys file on the new VM, or by temporarily enabling password authentication. Verify the SSH connectivity before starting the transfer below.
  • Sync your data: Use rsync to securely copy your files and directories directly over the network:
rsync -avz --progress /path/to/old/data/ <username>@<NEW_VM_IP>:/path/to/new/destination/
🛠️ Tips and Tricks:
  • rsync uses SSH underneath, and SSH will automatically try the keys available to the current user (e.g., ~/.ssh/id_ed25519, ~/.ssh/id_rsa, keys loaded into ssh-agent, etc.). If you use a non-default SSH key, you can specify it with rsync -e "ssh -i </path/to/private-key>".
  • Make sure that the remote user has write permission for the destination directory.
  • For IPv6-addresses, enclose them in square brackets [].

Transferring VMs (not recommended)

Prerequisites

  • Access to both bwCloud-OS environments and
  • Application Credentials, as described here and
  • OpenStack Client installed, as described here.

Procedure

When transferring a VM, you create a snapshot of its current state, download it, and upload and use it as a base image in the new environment.

Step 1: Create a snapshot

Source your old bwCloud-OS Gen2 credentials and find the ID of the instance (VM) you want to migrate, then create a snapshot of it as an image:

source old-creds.sh 
openstack server list 
openstack server image create --name <snapshot_name> <instance_id>

Step 2: Download the image

Once the image state is changed from queued to active, you can download it to your computer:

openstack image list
openstack image save --file myimage.img <snapshot_id> 

Step 3: Upload the image to the new environment (bwCloud-OS Gen3)

Source your new cloud credentials and upload the image file. Important: Set the visibility to private so that other users cannot use your image.

source new-creds.sh
openstack image create --file myimage.img --private myimage
⚠️ Please be aware that the upload process can take a significant amount of time depending on your local internet connection. For instance, uploading a 12 GB image may take anywhere from 30 to 60 minutes. Please be patient and do not interrupt the process.
⚠️ While it is technically possible to upload the downloaded image file via the Horizon Dashboard (by navigating to Compute -> Images), this method is highly prone to errors, especially with large image files. We strongly advise avoiding the dashboard for this task and using the CLI method described above instead to prevent corrupted or interrupted uploads.


Step 4: Start the instance

You can now start a new instance using this uploaded image. To do so:

  • Log in to the bwCloud-OS Gen3 Dashboard.
  • Navigate to Project -> Compute -> Images. The uploaded snapshot should be listed there.
  • Click on "Launch" (or "Start"), choose an instance name, and select the appropriate flavor, network, and SSH key.


🛠️ Possible Post-Migration Adjustments:

Once your instance is running, please be aware of the following necessary checks and potential reconfigurations:

  • Default Usernames: The SSH username is tied to the migrated image. If your Gen2 image used a specific default username (for example, on Opensuse images), the migrated Gen3 instance will retain that exact username. This might differ from the standard usernames used by fresh Gen3 images, so please ensure you use the correct login credentials.
  • SSH Keys: Similar to the username, the new Gen3 environment might not successfully inject newly selected SSH keys into a migrated image. If you cannot log in with the newly assigned key, you will need to use your old Gen2 private SSH key to access the VM.
  • Network Configuration & Security: You will receive a new IP address and FQDN, and Security Groups do not migrate. Therefore, you must manually recreate your (non-default) Security Groups in the Gen3 dashboard. Additionally, depending on the specific software running on your VM, you might need to update internal, network-dependent settings. If applicable, remember to adjust internal firewall rules, update Let's Encrypt certificates, or modify static IP configurations within your applications.
  • DNS Resolution: If you experience issues with domain name resolution within your VM after the migration, check your VM's DNS configuration (e.g., /etc/resolv.conf on many Linux systems). You may need to remove old Gen2 DNS server entries to restore functionality.