Storage: Difference between revisions

From bwCloud-OS
Jump to navigation Jump to search
No edit summary
No edit summary
Line 83: Line 83:
= Performance =
= Performance =


== Throttling of Data Throughput ==
== Is  storage throughput (IOPS) in the bwCloud-OS throttled? ==


Due to the internal architecture, all data (root disks of instances, attached storage volumes, etc.) resides in the Ceph storage of the respective region. Ceph is a network-based distributed storage system connected to the compute hosts via the network.. The available storage throughput is therefore shared among all active users. The more parallel write operations there are, the lower the throughput for each individual. This is logical, as the network capacity and bandwidth are limited overall.
Yes. Storage performance in bwCloud-OS is '''intentionally throttled''' to ensure fair usage across all users.


To provide roughly equal performance to all users, the storage throughput per instance is limited to either 100 MB/s in both directions (full duplex) or 800 IOPS.
All data — including root disks and attached volumes — is stored in the '''Ceph distributed storage system''', which is connected to the compute hosts via the internal network. This means that the storage performance depends on shared network resources''':''' The more simultaneous activity there is, the more the available bandwidth is divided among users.


== Requesting Higher Data Throughput ==
To ensure fair access, each instance is limited to a '''maximum of 100 MB/s''' (full duplex) or '''800 IOPS.'''


If justified, users can request higher data throughput. Please submit a [https://bw-support.scc.kit.edu/ ticket] to us. In the ticket, include the following information:
This limitation helps maintain consistent performance in a multi-user environment and prevents individual VMs from monopolizing bandwidth.


* You would like to receive higher data throughput.
== Can I request higher storage throughput in bwCloud-OS? ==
* Description of the use case or application: Why do you need higher throughput?
* OpenStack identifier (ideally the user ID).
* In which region do you need higher throughput?


The above points are mandatory.
Yes, in justified cases, higher data throughput can be granted.
 
Please submit a [https://bw-support.scc.kit.edu/ support ticket] , including the following mandatory details:
 
* A short note that you are requesting '''higher storage throughput'''
* A description of your '''use case or application''' and why the increased throughput is needed
* Your '''OpenStack user ID'''
* The '''region''' where the increased throughput is required

Revision as of 13:47, 19 October 2025

In a Nutshell
  • If you need more storage, create and attach additional volumes — root disks cannot be enlarged. Using a larger root disk via special flavors is also not recommended in general.
  • All data is stored redundantly (e.g., 3× replication in Ceph), but no automatic backups are performed — you are responsible for your own data protection.
  • Back up your data regularly using snapshots or by downloading volumes/images via the OpenStack client as explained here.


Volumes (Block Devices)

What should I do if the root disk of my VM is/becomes too small?

Quickstart: attach volume

Backups

Does bwCloud-OS provide a dedicated interface for backups or data protection?

No, bwCloud-OS does not offer a dedicated interface for backups or data protection. However, you can back up your instance data using standard tools and methods, see below.

How can I back up my virtual machines?

You can back up your VMs in bwCloud-OS in several ways:

🔹 Snapshots via the Dashboard

  • You can create snapshots of both instances and attached volumes using the web interface.
  • Instance snapshots are saved as images and can be downloaded using the CLI, see here for instructions.
  • You can also convert volumes into images and download them the same way, see here for instruction.

🔹 Export data directly from the instance

For large volumes or more control, it's often easier and more efficient to back up data directly from inside the VM using standard tools such as rsync, scp, or other backup utilities.

How can I export or import a (volume-)image via OpenStack CLI?

Connect to bwCloud-OS using the OpenStack Client.

For volumes, first create an image from the volume:

# openstack volume list
# openstack image create \
    --volume <UUID> \
    my_volume_as_image

Download a image:

# openstack image list
# openstack image save \
    --file my_image_file.img \
    <UUID>

Upload a local image file to the image catalog of the selected region and create its metadata entry:

# openstack image create \
    --property os_distro=linux \
    --property ssh_user=<USER> \
    --property hw_video_model=cirrus \
    --file my_image_file.img \
    <NAME>

How secure is my data (in terms of integrity) in bwCloud-OS?

Both the system disk (root disk) of a virtual machine and any attached volumes are stored redundantly, either in our distributed Ceph storage systems (most regions) or another redundant storage system (Freiburg). Each piece of data has redundancy level 3, which provides robust protection against hardware failure.

However, please note:

  • No regular backups of VM data are performed. Neither the root disk nor attached volumes are backed up by default.
  • It is your responsibility to ensure that important data is backed up internally (e.g. via snapshots) and externally (using external backup systems).

⚠️ Important Operational Note

bwCloud-OS is operated as a "best-effort" infrastructure service. This means:

  • To maintain performance and affordability, the system does not include mechanisms such as geo-redundancy.
  • In very rare cases (e.g. multiple simultaneous disk failures or critical software bugs in Ceph), data loss may occur and recovery may not be possible.

For this reason, we strongly recommend storing critical or irreplaceable data (including essential configuration files and recovery information) on external storage systems outside the bwCloud-OS.

Performance

Is storage throughput (IOPS) in the bwCloud-OS throttled?

Yes. Storage performance in bwCloud-OS is intentionally throttled to ensure fair usage across all users.

All data — including root disks and attached volumes — is stored in the Ceph distributed storage system, which is connected to the compute hosts via the internal network. This means that the storage performance depends on shared network resources: The more simultaneous activity there is, the more the available bandwidth is divided among users.

To ensure fair access, each instance is limited to a maximum of 100 MB/s (full duplex) or 800 IOPS.

This limitation helps maintain consistent performance in a multi-user environment and prevents individual VMs from monopolizing bandwidth.

Can I request higher storage throughput in bwCloud-OS?

Yes, in justified cases, higher data throughput can be granted.

Please submit a support ticket , including the following mandatory details:

  • A short note that you are requesting higher storage throughput
  • A description of your use case or application and why the increased throughput is needed
  • Your OpenStack user ID
  • The region where the increased throughput is required