OpenStack Summit November 2014 Paris

Ask the Experts: Challenges for OpenStack Storage 

Here's your chance! This all-star panel discussion will focus on the challenges for OpenStack storage while responding directly to feedback from users and developers in the audience of their challenges with OpenStack storage. This panel may be recorded as a "Speaking in Tech" podcast which is distributed by Europe's largest tech publication, The Register.

Topics that will be discussed include:

• Feedback directly from audience members about their challenges with OpenStack storage







• How data protection is evolving in OpenStack







• Panel feedback on the challenges with OpenStack storage







• What features are missing from OpenStack?







• Managing support for OpenStack storage

Panelists:

Mark Twomey (aka @Storagezilla)

Technical Director in the CTO team for EMC's Data Protection & Availability Division

Alex McDonald

NetApp Industry Forums & Standards, CTO Office

Manju Ramanthpura

CTO, Intelligent Platforms, Hitachi Data Systems

Neil Levine

Director of Product Management, Red Hat

Aaron Delp

Cloud Solutions Architect, SolidFire

 
Moderator:

Greg Knieriemen

Technology Evangelist, Hitachi Data Systems and...

Co-host of the "Speaking in Tech" Podcast on The Register

In this presentation we dive deep into OpenStack Trove, the Database-as-a-Service project in OpenStack. The goal of Trove is to provide a scalable and reliable Cloud Database as a Service provisioning functionality for both relational and non-relational database engines and to continue to improve its fully featured and extensible open source framework.

The presentation will describe the architecture and internals of Trove in detail and provide an in-depth view of each of these components. We will also provide some historical perspective and chart the evolution of the Trove project as it has evolved over the past 18 or so months as many of the architectural decision and trade-offs have been made in the process of this evolution, and to truly understand the architecture one must understand some of the history as well. We will also provide an overview of the layout of the source code of the project and highlight the location of the various major architectural elements in the source repository.

The presentation is valuable to developers and aspiring contributors to OpenStack, especially Trove, as it will give them a very solid understanding of the project and the source code and make it much easier for them to begin to contribute to the project at a very fundamental level. The presentation is also valuable for OpenStack adopters and those interested in operating databases in the OpenStack cloud. It will help decision makers understand the flexibility of the Trove architecture and understand why choosing Trove, as the basis for a Database-as-a-Service, is a very good one.

Following the release of Swift 2.0, the community has been working feverishly on the first significant Storage Policy for Swift Erasure Code support.  This is talk will certainly touch on the value of this feature to deployers but really focus more on educating the audience on exactly how it is that Erasure Codes work to enable such a dramatic cost savings while preserving or increase durability.

Additionally, this talk will take a close look at the options that the community faced as it examined implementation choices including what we came up with any why.  We will provide an overview of a OpenStack requirement, a Python Erasure Code Library known as PyECLib supporting the most popular and efficient algorithms out there and explain how we leveraged Storage Policies to add Erasure Code support without turning the codebase upside down.

Come hear the story directly from two of the main Erasure Code contributors from Box and Intel!


 

Additionally, this talk will take a close look at the options that the community faced as it examined implementation choices including what we came up with any why.  We will provide an overview of a OpenStack requirement, a Python Erasure Code Library known as PyECLib supporting the most popular and efficient algorithms out there and explain how we leveraged Storage Policies to add Erasure Code support without turning the codebase upside down.

 

Come hear the story directly from two of the main Erasure Code contributors from Box and Intel!

The most frequent problem I see from customers is 'I have configured Cinder but my volumes just go to 'ERROR' state.  Why?'  There are a number of common pitfalls that storage administrators who are new to Cinder may encounter.  The source of the problem may not be easily determined by new users, but with a bit of guidance most problems are easily resolved.  This  presentation will help guide new, and existing, users to smooth attainment of an 'available' volume on new storage back-ends whether it be on a new system or an additional storage system being added via Cinder's multi-backend support.

What is the relationship between Cloud Computing and what has been termed, the Software Defined Data Center (SDDC)? Does OpenStack already implement Software Defined Storage (SDS) and if not, what should be the role of SDS in the OpenStack platform? In this panel, we will hear from storage experts with a diversity of opinions on these important questions. We will have a discussion on the role of SDS of in OpenStack broadly and the implementation of SDS being proposed by vendors such as EMC and Huawei. You will hear from proponents as well critics who believe that the current vendor proposals for SDS are at odds with the open source vision for OpenStack. The goal is to begin driving an open and honest discussion about the future of OpenStack storage technologies.

Openstack Swift is a very powerful object storage that is used in several of the largest object storage deployments around the globe. It ensures a very high level of data durability and can withstand epic disasters if setup in the right way.

 

During this talk we want to give you an overview about the mechanism that are used within Swift to ensure data durability and availability, how to design your cluster using regions and zones and things you need to pay attention to, especially when enhancing an existing cluster.

 

This talk will deal with performance related issues that operators may have to face when updating an existing Swift cluster. Such changes may require big volumes of production data to flow between nodes, which can have an impact on the cluster s performance, not to mention the guarantee that no data is lost during the operation. In order to achieve such migration while ensuring good performance to Swift users a number of parameters have to be tuned. Monitoring tools will also play a role to ensure that your cluster behaves appropriately.

 

The talk will be concluded with an estimation of the data durability based on your Swift configuration.

Swift is an object storage system designed for data that needs to be instantly accessible, stored forever, and accessible from multiple devices. In short, it is the engine that runs the biggest storage clouds on the planet. This session will cover Swift's architecture, its technology capabilities, and real-world infrastructure-as-a-service use cases. This OpenStack Swift introduction is great for attendees who want to understand the design goals of Swift and how they can best make use of this OpenStack component. It will be an informative introduction for those interested in running Swift or contributing to the Swift project.

NTT Data is an IT service company which especially works on infrastructure systems.

In this talk, we introduce our recent big project about OpenStack swift, for one of the Telecom companies in Japan. In this project, we constructed six swift clusters with multi-petabyte capacity, distributed over two sites.

We especially focus on following challenges we tackled in the project.
 * Automated Building and testing of swift clusters to ensure fast and safe delivery
 * Monitoring and visualization system for huge swift clusters to manager huge distributed swift clusters
 * Backup system from a multi-site swift cluster to another swift cluster to realize more robust data storing
 * Performance tuning for hardware with no SSD to realize effective performance with low cost hardware

We spend two years to build high-performance,elastic,high-durability block storage based on  OpenStack and Ceph, it  has been running for one year in our public cloud.

In this session, you learn details of:

• Provisioned performance. The performance of each volume is 5000 IOPS, 170MB/s throughput, 2ms latency, how do we archieve these SLA and QoS, how do we select hardware and software . how do we optimize IO stack components .

• High durability. The data durability is 99.99999999%, how do we design and configure the storage architecture to get the durability.

• Operation friendly. All operations are harmless for SLA, how do we expand the capacity and upgrade software and hardware.

OpenStack Swift is and has always been a modular, extensible storage system. Swift is not an abstraction layer on top of an object storage system. OpenStack Swift provides clients a unified namespace that abstracts away the underlying storage volumes.

With this simple concept, Swift gives operators seamless capacity management and provides storage for applications that just works, even when hardware fails. More importantly, it gives operators a simple storage service that can be tuned and extended to their exact needs functionally, financially, and over the lifetime of their applications. Swift can be extended by both middleware and by replacing the underlying storage volumes, and these extensibility points have exsited in the project since it began. This talk will demonstrate Swift's extensibility and reference both existing products extending Swift and discuss future possibilities for extension.

If you are interested in learning how Swift can be adapted to seamlessly fit into your environment, you won't want to miss this talk.