Storage: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
| (28 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
<span id="In-a-Nutshell"></span> | <span id="In-a-Nutshell"></span> | ||
{{InANutshell| | {{InANutshell| | ||
<li>If you need more storage in a VM, create and attach additional '''volumes''' — root disks cannot be | <li>If you need more storage in a VM, create and attach additional '''volumes''' — root disks cannot be resized. Using special flavors with larger root disks as a reserve for extra storage is generally discouraged, as they offer less flexibility. </li> | ||
<li>All data is stored redundantly (e.g., 3× replication in Ceph), but '''no automatic backups''' are performed — ''you are responsible for backing up your own data''.</li> | <li>All data is stored redundantly (e.g., 3× replication in Ceph), but '''no automatic backups''' are performed — ''you are responsible for backing up your own data''.</li> | ||
<li>Back up your data regularly using '''snapshots''' or by downloading (volume-)images using the OpenStack client as explained | <li>Back up your data regularly using '''snapshots''' or by downloading (volume-)images using the OpenStack client as explained | ||
[[Storage#How_can_I_back_up_my_virtual_machines?|here]].</li> | [[Storage#How_can_I_back_up_my_virtual_machines?|here]].</li> | ||
<li>⚠️''To avoid data loss: '''Never store critical or irreplaceable data only in bwCloud-OS ''' — always back up externally''.<li> | <li>⚠️ ''To avoid data loss: '''Never store critical or irreplaceable data only in bwCloud-OS ''' — always back up externally''.<li> | ||
}} | }} | ||
| Line 13: | Line 13: | ||
= Volumes (Block Devices) = | = Volumes (Block Devices) = | ||
== What should I do if the root disk of my VM | == What should I do if the root disk of my VM becomes too small? == | ||
<span id="Volume-Attach"></span> | |||
The size of the root disk (system disk) cannot be increased after the VM is created. If you need more storage, the recommended solution is to attach additional volumes (virtual disks) to your instance. | |||
If your current quota is not sufficient to create the needed volumes, please submit a ticket to [[Projects and Quota#Quota-Increase-Request|increase your volume quota]] via the [https://bw-support.scc.kit.edu bwSupportPortal]. | |||
The following guide walks you through all required operations. | |||
[[Guide: Volumes and Images#Create and Attach a Volume|➡️ Volumes and Images Guide]] | |||
= Backups = | = Backups = | ||
== Does bwCloud-OS provide a dedicated interface for backups | == Does bwCloud-OS provide a dedicated interface for backups? == | ||
<span id="Backup-Interface"></span> | |||
No, bwCloud-OS does not offer a dedicated interface for backups or data protection. However, you can back up your instance data using standard tools and methods, see below. | No, bwCloud-OS does not offer a dedicated interface for backups or data protection. However, you can back up your instance data using standard tools and methods, see below. | ||
== How can I back up my virtual machines? == | == How can I back up my virtual machines? == | ||
<span id="Backup-VM"></span> | |||
You can back up your VMs in bwCloud-OS in several ways: | You can back up your VMs in bwCloud-OS in several ways: | ||
🔹 '''Snapshots via the Dashboard''' | 🔹 '''Snapshots via the Dashboard''' | ||
* You can create snapshots of both instances and | * You can create snapshots of both instances and volumes using the web interface. | ||
* Instance snapshots are saved as images and can be downloaded using the CLI, see [[ | * Instance snapshots are saved as images and can be downloaded using the CLI, see [[#Image-Export-Import|here]] for instructions. | ||
* You can also convert volumes into images and download them the same way, see [[ | * You can also convert volumes into images and download them the same way, see [[#Image-Export-Import|here]] for instructions. | ||
🔹 '''Export data directly from the instance''' | 🔹 '''Export data directly from the instance''' | ||
For large volumes or more control, it's often easier and more efficient to back up data directly from inside the VM using standard tools such as <code>rsync</code>, <code>scp</code>, or other backup utilities. | *For large volumes or more control, it's often easier and more efficient to back up data directly from inside the VM using standard tools such as <code>rsync</code>, <code>scp</code>, or other backup utilities. | ||
== | ==Can I export or import a (volume-)image? == | ||
<span id="Image-Export-Import"></span> | |||
Yes, that is possible. However, you should use the [[Programmatic_Access_and_Automation#OpenStack_Client|OpenStack CLI Client]]. The main commands can be found in the following guide. | |||
[[Guide: Volumes and Images#Export and Import (Volume-)Images|➡️]] [[Guide: Volumes and Images#Exporting and Importing (Volume-)Images|Volumes and Images Guide]] | |||
# | |||
# | |||
== How secure is my data (in terms of integrity) in bwCloud-OS? == | |||
<span id="Data-Integrity"></span> | |||
< | |||
</ | |||
Both the system disk (root disk) of a virtual machine and any attached volumes are stored redundantly, either in our distributed '''Ceph storage systems''' (most regions) or another redundant storage system (Freiburg). Each piece of data has redundancy level 3, which provides robust '''protection against individual hardware failures'''. | |||
Both the system disk (root disk) of a virtual machine and any attached volumes are stored redundantly, either in our distributed '''Ceph storage systems''' (most regions) or another redundant storage system (Freiburg). Each piece of data has redundancy level 3, which provides ''' | |||
However, please note: | However, please note: | ||
* '''No regular backups''' of VM data are performed. | * '''No regular backups''' of VM data are performed. By default, neither the root disk nor attached volumes are backed up. | ||
* It is '''your responsibility''' to ensure that important data is backed up internally (e.g. via snapshots) and externally (using external backup systems). | * It is '''your responsibility''' to ensure that important data is backed up internally (e.g. via snapshots) and externally (using external backup systems). | ||
| Line 78: | Line 68: | ||
* To maintain performance and affordability, the system does not include mechanisms such as geo-redundancy. | * To maintain performance and affordability, the system does not include mechanisms such as geo-redundancy. | ||
* In very rare cases (e.g. multiple simultaneous disk failures or critical software bugs in Ceph), data loss may occur and recovery may not be possible. | * In very rare cases (e.g., multiple simultaneous disk failures or critical software bugs in Ceph), data loss may occur and recovery may not be possible. | ||
''For this reason, we '''strongly recommend''' storing '''critical or irreplaceable data''' (including essential configuration files and recovery information) on '''external storage systems''' '''outside the bwCloud-OS'''.'' | ''For this reason, we'' ''strongly recommend'' ''storing '''critical or irreplaceable data''' (including essential configuration files and recovery information) on '''external storage systems''' '''outside the bwCloud-OS'''.'' | ||
= Storage Performance = | = Storage Performance = | ||
== Is storage throughput in the bwCloud-OS throttled? == | == Is storage throughput in the bwCloud-OS throttled? == | ||
<span id="Throughput"></span> | |||
Yes. Storage performance in bwCloud-OS is '''intentionally throttled''' to ensure fair usage across all users. | Yes. Storage performance in bwCloud-OS is '''intentionally throttled''' to ensure fair usage across all users. | ||
| Line 90: | Line 81: | ||
All data — including root disks and attached volumes — is stored in the '''Ceph distributed storage system''', which is connected to the compute hosts via the internal network. This means that the storage performance depends on shared network resources''':''' The more simultaneous activity there is, the more the available bandwidth is divided among users. | All data — including root disks and attached volumes — is stored in the '''Ceph distributed storage system''', which is connected to the compute hosts via the internal network. This means that the storage performance depends on shared network resources''':''' The more simultaneous activity there is, the more the available bandwidth is divided among users. | ||
To ensure fair access, each instance is limited to a '''maximum of 100 MB/s''' (full duplex) | To ensure fair access, each instance is limited to a '''maximum of 100 MB/s''' (full duplex) and '''800 IOPS.''' | ||
This limitation helps maintain consistent performance in a multi-user environment and prevents individual VMs from monopolizing bandwidth. | This limitation helps maintain consistent performance in a multi-user environment and prevents individual VMs from monopolizing bandwidth. | ||
== Can I request higher storage throughput in bwCloud-OS? == | == Can I request higher storage throughput in bwCloud-OS? == | ||
<span id="Throughput-Increase"></span> | |||
Yes, in justified cases, higher data throughput can be granted. | Yes, in justified cases, higher data throughput can be granted. | ||
Please submit a [https://bw-support.scc.kit.edu/ support ticket], including the following | Please submit a [https://bw-support.scc.kit.edu/ support ticket], including the following required information: | ||
* A short note that you are requesting '''higher storage throughput''' | * A short note that you are requesting '''higher storage throughput''' | ||
Latest revision as of 09:22, 22 October 2025
| In a Nutshell |
|
Volumes (Block Devices)
What should I do if the root disk of my VM becomes too small?
The size of the root disk (system disk) cannot be increased after the VM is created. If you need more storage, the recommended solution is to attach additional volumes (virtual disks) to your instance.
If your current quota is not sufficient to create the needed volumes, please submit a ticket to increase your volume quota via the bwSupportPortal.
The following guide walks you through all required operations.
Backups
Does bwCloud-OS provide a dedicated interface for backups?
No, bwCloud-OS does not offer a dedicated interface for backups or data protection. However, you can back up your instance data using standard tools and methods, see below.
How can I back up my virtual machines?
You can back up your VMs in bwCloud-OS in several ways:
🔹 Snapshots via the Dashboard
- You can create snapshots of both instances and volumes using the web interface.
- Instance snapshots are saved as images and can be downloaded using the CLI, see here for instructions.
- You can also convert volumes into images and download them the same way, see here for instructions.
🔹 Export data directly from the instance
- For large volumes or more control, it's often easier and more efficient to back up data directly from inside the VM using standard tools such as
rsync,scp, or other backup utilities.
Can I export or import a (volume-)image?
Yes, that is possible. However, you should use the OpenStack CLI Client. The main commands can be found in the following guide.
How secure is my data (in terms of integrity) in bwCloud-OS?
Both the system disk (root disk) of a virtual machine and any attached volumes are stored redundantly, either in our distributed Ceph storage systems (most regions) or another redundant storage system (Freiburg). Each piece of data has redundancy level 3, which provides robust protection against individual hardware failures.
However, please note:
- No regular backups of VM data are performed. By default, neither the root disk nor attached volumes are backed up.
- It is your responsibility to ensure that important data is backed up internally (e.g. via snapshots) and externally (using external backup systems).
⚠️ Important Operational Note
bwCloud-OS is operated as a "best-effort" infrastructure service. This means:
- To maintain performance and affordability, the system does not include mechanisms such as geo-redundancy.
- In very rare cases (e.g., multiple simultaneous disk failures or critical software bugs in Ceph), data loss may occur and recovery may not be possible.
For this reason, we strongly recommend storing critical or irreplaceable data (including essential configuration files and recovery information) on external storage systems outside the bwCloud-OS.
Storage Performance
Is storage throughput in the bwCloud-OS throttled?
Yes. Storage performance in bwCloud-OS is intentionally throttled to ensure fair usage across all users.
All data — including root disks and attached volumes — is stored in the Ceph distributed storage system, which is connected to the compute hosts via the internal network. This means that the storage performance depends on shared network resources: The more simultaneous activity there is, the more the available bandwidth is divided among users.
To ensure fair access, each instance is limited to a maximum of 100 MB/s (full duplex) and 800 IOPS.
This limitation helps maintain consistent performance in a multi-user environment and prevents individual VMs from monopolizing bandwidth.
Can I request higher storage throughput in bwCloud-OS?
Yes, in justified cases, higher data throughput can be granted.
Please submit a support ticket, including the following required information:
- A short note that you are requesting higher storage throughput
- A description of your use case or application and why the increased throughput is needed
- Your OpenStack user ID
- The region where the increased throughput is required