Search for notes by fellow students, in your own course and all over the country.
Browse our notes for titles which look like what you need, you can preview any of the notes via a sample of the contents. After you're happy these are the notes you're after simply pop them into your shopping cart.
Document Preview
Extracts from the notes are below, to see the PDF you'll receive please use the links above
Red Hat OpenShift Container Platform 4
...
1
Design Guide
Abstract
This design guide provides architecture and design information for the
Dell Technologies Validated Design for Red Hat® OpenShift® Container
Platform 4
...
Dell Technologies Solutions
Copyright
The information in this publication is provided as is
...
makes no representations or warranties of any kind with respect
to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular
purpose
...
Copyright © 2022 Dell Inc
...
All Rights Reserved
...
or its subsidiaries
...
S
...
Other trademarks may be trademarks of their respective owners
...
1
...
believes the information in this document is accurate as of its publication date
...
2
Red Hat OpenShift Container Platform 4
...
6
Document purpose
...
8
We value your feedback
...
10
Red Hat OpenShift Container Platform
...
13
Server options
...
19
Deployment process
...
22
Chapter 3
Networking Infrastructure and Configuration
25
Introduction
...
26
Physical network design
...
35
OpenShift Data Foundation
...
40
CSI external storage
...
43
Chapter 5
Cluster Hardware Design
44
Introduction
...
45
Cluster hardware planning
...
47
Chapter 6
Use Cases
51
Overview
...
52
Networking
...
57
Red Hat OpenShift Container Platform 4
...
61
Red Hat documentation
...
62
Appendix A Hardware Configuration
63
Overview
...
64
Dell PowerEdge R740xd node BOM
...
66
Dell PowerEdge R750 BOM
...
67
Dell Unity 380F BOM
...
69
OpenShift Data Foundation data node configurations
...
6 on Dell EMC Infrastructure
Design Guide
Chapter 1: Introduction
Chapter 1 Introduction
This chapter presents the following topics:
Solution overview
...
7
Audience
...
8
Red Hat OpenShift Container Platform 4
...
6 can host the development and run-time
execution of containerized applications
...
The Dell Technologies Validated Design (DTVD) for OpenShift Container Platform 4
...
6 on-premises bare-metal deployment
...
The emphasis is on running cloud-native
applications on containers that are managed by OpenShift Container Platform
...
•
A selection of validated OpenShift Container Platform hardware designs
•
A scalable hardware platform of up to 210 compute nodes spread across seven
racks
•
Rapid implementation and reduced time to value
This version of the DTVD for Red Hat OpenShift Container Platform 4
...
Change history
Publication date
New features
January 2022
Support for:
•
October 2021
6
Red Hat OpenShift Container Platform 4
...
6 on Dell EMC Infrastructure
Implementation Guide: Information about automation-assisted deployment of the
solution
...
6 on Dell EMC Infrastructure Design
Guide: This document
...
6 documents provide information about:
•
Manual installation and deployment of Red Hat software products
•
Platform deployment preferences OpenShift Container Platform 4
...
Note: While you can rely on Red Hat Enterprise Linux security and container technologies to
prevent intrusions and protect your data, some security vulnerabilities might persist
...
For a
general listing of Red Hat vulnerabilities, see the Red Hat Security Home Page
...
6
...
•
Extend or modify the design as necessary to meet customer requirements
...
6 for application development and
deployment
•
Dell PowerEdge R640 and R740xd servers for compute and storage
Red Hat OpenShift Container Platform 4
...
Some experience
with containers, Kubernetes, and the OpenShift Container Platform is recommended
...
Contact the Dell Technologies Solutions team by
email with your comments
...
When this content is updated
by the relevant third parties, this guide will be revised accordingly
...
6 on Dell EMC Infrastructure
Design Guide
Chapter 2: Technology and Deployment Process Overview
Chapter 2 Technology and Deployment
Process Overview
This chapter presents the following topics:
Introduction
...
10
Cloud-native infrastructure
...
18
Validated firmware
...
20
Infrastructure requirements
...
6 on Dell EMC Infrastructure
Design Guide
9
Chapter 2: Technology and Deployment Process Overview
Introduction
Overview
This chapter describes the OpenShift Container Platform architecture, infrastructure
components, and requirements of a viable architecture for a Red Hat OpenShift Container
Platform 4
...
This chapter
also describes the Dell Technologies simplified bootstrapping process for an OpenShift
Container Platform 4
...
Red Hat OpenShift Container Platform
Overview
OpenShift Container Platform is an enterprise-grade declarative state machine that has
been designed to automate application workload operations based on the upstream
Kubernetes project
...
OpenShift Container Platform uses the OpenShift
Kubernetes Engine, an enterprise-grade Kubernetes distribution, to provide productionoriented container and workload automation
...
6 is based
on Kubernetes version 1
...
Also, OpenShift Container Platform 4
...
Built on top of Kubernetes, OpenShift Container Platform gives administrators
and developers the tools that they need to deploy and manage applications and services
at scale
...
Certification for
Kubernetes distributions is provided by the Cloud Native Computing Foundation
...
6 on Dell EMC Infrastructure
Design Guide
Chapter 2: Technology and Deployment Process Overview
OpenShift Container Platform architecture
Kubernetes
concepts
In Kubernetes, everything is an object
...
This specification includes
everything from applications, deployments, and services to machine configuration and
management of specific hardware resources
...
Custom Resource Definitions (CRDs) can be used to specify new resource types, which
can then be used to create Custom Resources (CRs)
...
What Kubernetes is
Kubernetes provides an abstraction layer for application containers, deployments, and
services and automates all container operations
...
What Kubernetes is not
There is no imperative management of containers in Kubernetes
...
This mechanism has fundamental
Red Hat OpenShift Container Platform 4
...
Upstream Kubernetes has some fundamental limitations in that it does not build or deploy
applications; provide logging, monitoring, or alerting mechanisms; and is not a selfhealing, self-managing system
...
Why OpenShift
Container
Platform?
OpenShift Container Platform fills the gaps that Kubernetes leaves open, such as:
•
Platform-level services, including building and packaging applications
•
Integrated logging and monitoring solutions (Prometheus and Grafana)
•
Integrated web console
OpenShift Container Platform is intended as a turnkey solution for production-grade
environments
...
•
Provides additional self-management capabilities that are not found in Kubernetes
due to the tightly coupled toolchain: the default containers-first operating system
(Red Hat CoreOS), a Kubernetes-first container run-time (CRI-O), and a rigorous
testing and certification process for additional Red Hat and vendor middleware
...
Operators attempt to encode the operational knowledge that is required
for various stateful applications
...
Like Helm, a universal package
manager for Kubernetes, an operator can be used to configure and install middleware
...
Operators are designed to simplify Day-2 operations by automatically deploying, updating,
and maintaining specific application deployments
...
The following figure shows the benefits that operators can provide, depending on their
complexity:
12
Red Hat OpenShift Container Platform 4
...
These applications and components are
generally microservices-based
...
The reference
design that this guide describes helps you specify infrastructure requirements for building
an on-premises OpenShift Container Platform 4
...
The following figure shows this
design:
Red Hat OpenShift Container Platform 4
...
6 standard cluster design
OpenShift
Container
Platform
terminology
This architectural design recognizes four host types that make up every OpenShift
Container Platform cluster: the bootstrap node, the control-plane nodes, the compute
nodes, and the storage nodes
...
For a description of the deployment process, see the Red Hat OpenShift
Container Platform 4
...
Note: Red Hat official documentation does not refer to a CSAH node in the deployment process
...
CSAH nodes also provision DHCP, PXE, DNS,
and HAProxy services for cluster operation
...
For resilient load-balancing, Dell Technologies recommends using two CSAH
nodes running HAProxy and KeepAlived
...
The OpenShift CLI tool
called oc and the authentication tokens that are required to administer the OpenShift
cluster are installed on both CSAH nodes as part of the deployment process
...
Note: Control-plane nodes are deployed using immutable infrastructure, further driving the
preference for an administration host that is external to the cluster
...
6 on Dell EMC Infrastructure
Design Guide
Chapter 2: Technology and Deployment Process Overview
Bootstrap node
(VM)
The CSAH nodes manage the operation and installation of the container ecosystem
cluster
...
The initial
minimum cluster can consist of three nodes running both the control plane and
applications, or three control-plane nodes and at least two compute nodes
...
Basic node
configuration
Node components are installed and run on every node in the cluster; that is, on controlplane nodes and compute nodes
...
Key components consist of:
Control-plane
nodes
•
Kubelet: An agent that runs on each node to perform declarations or actions that
are provided to the cluster-API
...
Kubelet does
not manage containers or pods that were not created by Kubernetes
...
The
kube-proxy instance implements Kubernetes network services that run on the node
...
•
Container run-time: The chosen container run-time engine (CRE) must be
deployed on each node in a Kubernetes cluster
...
OpenShift Container
Platform defaults to the CRI-O container run-time and cannot be changed
...
Three control-plane nodes establish a unified control plane for the operation of an
OpenShift cluster
...
Removing control-plane nodes is not allowed
...
The control plane provides the following functions:
•
API Server: The API server exposes the Kubernetes control-plane API for other
platform services (such as a web console) to consume and has API endpoints to
manage cluster resources
...
The etcd daemon is run on each control plane node and
requires a majority consensus to achieve quorum (the formula used for quorum is
𝑛
+ 1, where n is the number of control plane nodes)
...
This
requirement means that at least two control planes are required to achieve quorum
...
Red Hat OpenShift Container Platform 4
...
While
each controller process is independent, the processes are run as a single
executable to reduce complexity
...
•
OpenShift API server: The OpenShift API server validates and configures the data
for OpenShift resources such as projects, routes, and templates
...
•
OpenShift controller manager: The OpenShift controller manager watches etcd
for changes to OpenShift objects such as project, route, and template controller
objects, and then uses the API to enforce the specified state
...
•
OpenShift OAuth API server: The OpenShift OAuth API server validates and
configures the data to authenticate to OpenShift Container Platform, such as users,
groups, and OAuth tokens
...
•
OpenShift OAuth server: Users request tokens from the OpenShift OAuth server
to authenticate themselves to the API
...
Backup and disaster recovery
Even though OpenShift Container Platform is resilient to node failure, it is recommended
to take regular backups of the etcd data store
...
Keep in mind that
when you update a cluster within minor versions (for example, from 4
...
2 to 4
...
3), you
should take an etcd backup of the version of OpenShift Container Platform that is
currently running on your cluster or clusters
...
For more information, see the Red Hat document
Backing up etcd
...
For the cluster restoration steps, see
Restoring to a previous cluster state
...
To perform this task, follow the steps in
Replacing an unhealthy etcd member
...
The term “compute node” is arbitrary; nothing specific is required to run compute
nodes, and applications can be run on control-plane nodes, if wanted
...
The
Kubelet service runs on all nodes in a Kubernetes cluster
...
The Kubelet service also starts and stops container workloads and manages a
service proxy that handles communication between pods that are running across compute
nodes
...
6 on Dell EMC Infrastructure
Design Guide
Chapter 2: Technology and Deployment Process Overview
Logical constructs called MachineSets define compute node resources
...
OpenShift Container Platform supports defining multiple machine types, each of which
defines a compute node target type
...
If the control-plane nodes are not designated as
schedulable, at least two viable compute nodes must always be operating to run router
pods that manage ingress networking traffic
...
Storage nodes
Storage can be either provisioned from dedicated nodes or shared with compute services
...
For more information, see Dell storage options
...
As part of the portfolio, Red Hat OpenShift Data Foundation (formerly
known as OpenShift Container Storage) delivers resilient and persistent SDS and data
services based on Ceph, Rook, and NooBaa technologies
...
OpenShift Data Foundation can be deployed within an OpenShift Container Platform
cluster on existing worker nodes, infrastructure nodes, or dedicated nodes
...
To streamline configuration options, Red Hat and Intel® have jointly developed
three workload-optimized configurations for OpenShift Data Foundation external data
nodes: edge, capacity, and performance
...
It is also possible to use existing
compute nodes if they meet OpenShift Data Foundation hardware requirements
...
For more information, see OpenShift Container Storage 4
...
Note: At the time of publication of this guide, some Red Hat documentation and the operator and
product interface of OpenShift Data Foundation may still use the product name OpenShift
Container Storage for OpenShift Data Foundation
...
6 on Dell EMC Infrastructure
Design Guide
17
Chapter 2: Technology and Deployment Process Overview
Server options
Dell Technologies has tested and fully validated the following PowerEdge server options
for an OpenShift Container Platform 4
...
The PowerEdge R650 is a two-socket/1U rack server that delivers
outstanding performance for the most demanding workloads
...
To address substantial throughput improvements, the PowerEdge R650 model
supports PCIe Gen 4 and up to 10 NVMe drives with improved air-cooling features plus
optional Direct Liquid Cooling to support increasing power and thermal requirements
...
PowerEdge R750 The PowerEdge R750 server, which is powered by the 3rd Generation Intel® Xeon®
server
Scalable processors, is a rack server to address application performance and
acceleration
...
The PowerEdge R750
supports eight channels of memory per CPU and up to 32 DDR4 DIMMs at 3200 MT/s
speeds
...
The
PowerEdge R750 is therefore an ideal server for data center standardization on a wide
range of workloads requiring performance, extensive storage, and GPU support
...
The scalable business architecture of the PowerEdge R640 platform is
designed to maximize application performance and provide you with the flexibility to
optimize configurations based on the application and use case
...
5” or 3
...
Combined with up to 24 DIMMs, 12 of which can be NVDIMMs, you have the
resources to create the optimum configuration to maximize application performance in
only a 1U chassis
...
The 2U two-socket platform is ideal for software-defined storage (SDS),
supporting up to 24 NVMe, or 32 x 2
...
5” drives
...
PowerEdge
XE2420 server
The PowerEdge XE2420 server is a configurable, two-socket, 2U rack server that delivers
powerful 2S performance in a short-depth form-factor
...
Its performance
can be further enhanced by its support of up to four accelerators
...
The
Red Hat OpenShift Container Platform 4
...
It is a front-accessible server that is designed to support
demanding edge applications such as streaming analytics, manufacturing logistics, and
5G cell processing
...
6 deployment using the PowerEdge R640 and R740xd,
the XE2420, and the R650 and R750 server platforms respectively:
Table 2
...
10
...
40
...
00
Mellanox ConnectX-4
14
...
45
...
28
...
12
Intel® XXV710
20
...
17
BOSS-S1
2
...
13
...
17
...
00
PERC H730P Mini controller
25
...
8
...
PowerEdge XE2420 tested firmware
Product
Version
BIOS
1
...
1
iDRAC with Lifecycle Controller
4
...
10
...
5
...
5
...
3024
HBA330 Mini
16
...
01
...
PowerEdge R650 and R750 tested firmware
Product
Version
BIOS
1
...
5
iDRAC with Lifecycle Controller
5
...
00
...
31
...
06
PERC H755N Front
52
...
1-4158
BOSS-S2
2
...
13
...
6 on Dell EMC Infrastructure
Design Guide
19
Chapter 2: Technology and Deployment Process Overview
Deployment process
Dell Technologies has simplified the process of bootstrapping the OpenShift Container
Platform 4
...
To use the simplified process, ensure that:
•
The cluster is provisioned with network switches and servers
...
•
Internet connectivity has been provided to the cluster
...
6
...
Initial switch provisioning enables
preparation and installation of the CSAH node and consists of:
•
Installing Red Hat Enterprise Linux 8
•
Subscribing to the necessary repositories
•
Creating an Ansible user account
•
Cloning a GitHub Ansible playbook repository from the Dell ESG container
repository
•
Running an Ansible playbook to initiate the installation process
Dell Technologies has generated Ansible playbooks that fully prepare both CSAH nodes
...
6 cluster begins, the Ansible
playbook sets up a PXE server, DHCP server, DNS server, HAProxy, and HTTP server
...
The playbook creates ignition files to drive installation
of the bootstrap, control-plane, and compute nodes, and it also starts the bootstrap VM to
initialize control-plane components
...
Note: For enterprise sites, consider deploying appropriately hardened DHCP and DNS servers
and using resilient multiple-node HAProxy configuration
...
This guide provides CSAH
Ansible playbooks for reference only at the implementation stage
...
yaml file that is used to control
deployment of the bootstrap node
...
6 on Dell EMC Infrastructure Implementation Guide
...
6 on Dell EMC Infrastructure
Design Guide
Chapter 2: Technology and Deployment Process Overview
Installation workflow: Creating the bootstrap, control-plane, and compute
nodes
Note: An installation that is driven by ignition configuration generates security certificates that
expire after 24 hours
...
The cluster bootstrapping process consists of the following phases:
1
...
Interrupting this process can prevent the OpenShift control
plane from coming up
...
The control-plane nodes pull resource information from the bootstrap VM to bring
them up to a viable state
...
3
...
4
...
5
...
6
...
7
...
8
...
9
...
The cluster is now viable and can be placed into service in readiness for Day-2
operations
...
Red Hat OpenShift Container Platform 4
...
6, two different types of cluster deployments are
available: a three-node cluster and a standard cluster (more than five nodes)
...
While the three-node cluster can be expanded with additional compute
nodes, an initial expansion of a three-node cluster requires that at least two compute
nodes be added simultaneously
...
If compute nodes are
added to a three-node cluster, deploy two compute nodes for full ingress functionality
...
A standard cluster deployment has
three control-plane nodes and at least two compute nodes
...
Both cluster deployment types require two CSAH nodes
for cluster management and resilient load-balancing
...
For detailed configuration information, see Cluster Hardware Design
...
Table 5
...
6 cluster deployment
Type
Description
CSAH node
Dell PowerEdge R640/R650
server
2
Control-plane
nodes
PowerEdge R640/R650 server
3
Compute nodes
PowerEdge R640 or R740xd
server
PowerEdge R650 or R750
server
Count
Notes
Creates a bootstrap VM
...
For
an enterprise high availability (HA)
deployment of OpenShift Container Platform
4
...
Options include commercial HAProxy,
Nginx, and F5
...
No compute nodes are required for a threenode cluster
...
To expand a three-node cluster, you must
add two compute nodes simultaneously
...
22
Red Hat OpenShift Container Platform 4
...
Dell PowerSwitch S5248ON
Note:
•
HA network configuration requires two
data path switches per rack
...
Leaf-spine network
switch configuration may be necessary
...
Rack
Selected according to site
standards
1–3 racks
For multirack configurations, consult your Dell
Technologies or Red Hat representative
regarding custom engineering design
...
To expand a three-node cluster with additional
compute machines, you must first expand the cluster to a five-node cluster using two additional compute nodes
...
While the cluster is in
production use, the CSAH node manages the cluster and performs load-balancing
for the cluster
...
•
Three nodes running both the control plane and data plane, enabling customers to
deploy OpenShift Container Platform 4
...
HA of the key services that make up the OpenShift Container Platform cluster is
necessary to ensure run-time integrity
...
In this design guide, HA includes the provisioning of at least two network interface
controllers (NICs) and two network switches that are configured to provide redundant
pathing
...
HA load-balancing can be provided by using an enterprise-grade load
balancer or an additional PowerEdge R640 server running HAProxy and KeepAlived
alongside the CSAH node
...
6 must use Red Hat Enterprise Linux CoreOS 4
...
Note: Using Red Hat Enterprise Linux 7 compute nodes is now deprecated and the ability to use
them in OpenShift will be removed in a future release of OpenShift
...
The bootstrap and control-plane nodes must use RHCOS 4
...
Each of these nodes must be immutable
...
6 on Dell EMC Infrastructure
Design Guide
23
Chapter 2: Technology and Deployment Process Overview
Table 6
...
6 nodes
Minimum CPU
cores
RAM
Storage
Red Hat Enterprise Linux
8
...
6
4
16 GB
120 GB
Controller
RHCOS 4
...
6
2
8 GB
120 GB
Node type
Operating system
CSAH
Network connectivity requirements
The RHCOS nodes must fetch ignition files from the Machine Config server
...
The startup requires a DHCP server to provide a network connection that gives access to
the ignition files for that node
...
24
Red Hat OpenShift Container Platform 4
...
26
OpenShift network operations
...
29
Red Hat OpenShift Container Platform 4
...
The reference design that this guide describes
uses Dell PowerSwitch networking infrastructure
...
Each compute node is equipped with resources such
as CPU cores, memory, storage, NICs, and add-in host adapters including GPUs,
SmartNICs, and FPGAs
...
The CNI API uses the Multus CNI plug-in to enable attachment of multiple adapter
interfaces on each pod
...
Container
communications
A pod, which is a basic unit of application deployment, consists of one or more containers
that are deployed together on the same compute node
...
As
service demand expands, additional identical pods are often deployed to the same or
other compute nodes
...
Four basic
network communication flows occur within every cluster:
•
Container-to-container connections (also known as highly coupled communication)
•
Pod communication over the local host network (127
...
0
...
Containers that communicate with any external pod originate their traffic based on the IP
address of the pod
...
Network traffic that
might be associated with nonlocal storage must be able to route across node network
infrastructure
...
Every node in a Kubernetes
cluster runs a kube-proxy and is responsible for implementing virtual IP (VIP) for services
...
•
Using DNS: OpenShift Container Platform 4
...
Red Hat OpenShift Container Platform 4
...
If the service uses HTTP, HTTPS, or any other TLS-encrypted
protocol, use an ingress controller; for other protocols, use a load balancer, external
service IP address, or node port
...
A service with
NodePort-type as a resource exposes the resource on a specific port on all nodes in
the cluster
...
Ingress
controller
OpenShift Container Platform uses an ingress controller to provide external access
...
Dell Technologies recommends creating a wildcard DNS entry and then setting
up an ingress controller
...
An ingress controller accepts external HTTP, HTTPS, and TLS
requests using SNI, and then proxies them based on the routes that are provisioned
...
Cluster IP routes
are created in the OpenShift Container Platform project, and a set of routes is admitted
into ingress controllers
...
Sharding enables you to:
Networking
operators
Container
Networking
Interface
•
Load-balance the incoming traffic
...
The following operators are available for network administration:
•
Cluster Network Operator (CNO): Deploys the OpenShift SDN plug-in during
cluster installation and manages kube-proxy on each node
...
•
Ingress operator: Enables external access to OpenShift Cluster Platform cluster
services and deploys and manages one or more HAProxy-based ingress controllers
to handle routing
...
The specification is used in both
upstream Kubernetes and OpenShift in the pod network
...
The most commonly used CNI plug-ins are:
•
Multus: CNI plug-in that supports the multinetwork function in Kubernetes
...
Multus acts as a
“meta plug-in,” a plug-in which calls other CNI plug-ins
...
•
DANM: Developed by Nokia, DANM is a CNI plug-in for telecom-oriented
workloads
...
6 on Dell EMC Infrastructure
Design Guide
27
Chapter 3: Networking Infrastructure and Configuration
for all Kubernetes hosts, and more
...
OpenShift SDN
OpenShift SDN creates an overlay network that is based on Open Virtual Switch (OVS)
...
OVS
operates in one of the following modes:
•
Network policy mode (the default), which allows custom isolation policies
•
Multitenant mode, which provides project-level isolation for pods and services
•
Subnet mode, which provides a flat network
OpenShift Container Platform 4
...
OVN-Kubernetes will become the default CNI
network provider in a future release of OpenShift Container Platform
...
6 supports additional SDN orchestration and management plug-ins that comply
with the CNI specification
...
Service Mesh
Distributed microservices work together to make up an application
...
The Red Hat OpenShift implementation of Service Mesh is based on Istio, an open-source
project
...
Service Mesh has key functional components that belong to either the data plane or the
control plane:
•
Envoy proxy: Intercepts all traffic for all services in Service Mesh
...
•
Mixer: Enforces access control and collects telemetry data
...
•
Citadel: Provides strong service-to-service and end-user authentication with
integrated identity and credential management
...
Users define the
granularity of the Service Mesh deployment, enabling them to meet their specific
deployment and application needs
...
Multinetwork
support
OpenShift Container Platform 4
...
The
platform comes with a default network
...
The additional
networks are useful for increasing the networking capacity of the pods and meeting traffic
separation requirements
...
Red Hat OpenShift Container Platform 4
...
•
Macvlan: Pods attached to a macvlan-based additional network have a unique
MAC address and communicate using a physical network interface
...
When pods are provisioned with additional network interfaces that are based on macvlan
or ipvlan, corresponding leaf-switch ports must match the VLAN configuration of the host
...
Physical network design
Design
principles
Dell networking products are designed for ease of use and to enable resilient network
creation
...
6 introduces various advanced networking
features to enable containers for high performance and monitoring
...
•
Configure dual-homing of the OpenShift Container Platform node to two Virtual Link
Trunked (VLT) switches
...
•
Allows monitor and trace container communications
...
Also, to meet network capacity requirements, pods can attach
to more networks by using available CNI plug-ins
...
Networking
applications such as Container Network Functions (CNFs) have control traffic and data
traffic
...
Pods can be attached to the SR-IOV virtual function (VF) interface on the host system for
traffic isolation and to increase I/O performance
...
The switches require VLT connections so
that they operate together as a single unit of connectivity to provide redundant data paths
for all network traffic
...
Red Hat OpenShift Container Platform 4
...
Dell
Technologies recommends deploying a leaf-spine network
...
6 supports Service Mesh
...
Resilient
networking
Each server that has many NIC options in the rack is connected to:
•
Two leaf switches with a network interface of choice: 10 GbE, 25 GbE, or 100 GbE
•
A management switch (typically, 1 GbE) for iDRAC connectivity
Our network design employs a VLT connection between the two leaf switches
...
VLT technology allows a server to uplink multiple physical trunks into more than one
PowerSwitch switch by treating the uplinks as one logical trunk
...
Both links from the bridge network
can forward and receive traffic
...
The major benefits of VLT technology are:
•
Dual control plane for highly available, resilient network services
•
Full utilization of the active link aggregation (LAG) interfaces
•
Active/active design for seamless operations during maintenance events
The VLTi configuration in this design uses two 100 GbE ports between each ToR switch
...
Scale out with
leaf-spine fabric
You can scale container solutions by adding multiple compute nodes and storage nodes
...
To create a nonblocking fabric that
meets the needs of the microservices data traffic, we used a leaf-spine network
...
•
Servers, storage arrays, edge routers, and similar devices connect to leaf switches,
but never to spines
...
We employed VLT in the spine
layer, which allows all connections to be active while also providing fault tolerance
...
The total number of leaf-spine connections is equal to the number of leaf switches
multiplied by the number of spine switches
...
6 on Dell EMC Infrastructure
Design Guide
Chapter 3: Networking Infrastructure and Configuration
the fabric by adding connections between leaf switches and spine switches if the spine
layer has capacity for the additional connections
...
The Layer 3-Layer 2 boundary is at the leaf switches
...
Equal cost multipath routing (ECMP) is used to loadbalance traffic across the Layer 3 network
...
Connections to external networks are made from a pair of edge or
border leaf switches, as shown in the following figure:
Leaf-spine network configuration
PowerSwitch
configuration
Dell’s high-capacity network switches are cost-effective and easy to deploy
...
OS10 allows
multilayered disaggregation of network functions that are layered on an open-source
Linux-based operating system
...
Red Hat OpenShift Container Platform 4
...
Enable Spanning Tree (the default) on the VLT peer switches
...
2
...
3
...
4
...
5
...
Installation with Ansible
Dell networking modules are supported in Ansible core from Ansible 2
...
You can use
these modules to manage and automate Dell switches running OS10
...
For an example of a Clos fabric deployment based on the Border Gateway Protocol
(BGP), see Provision CLOS fabric using Dell EMC Networking Ansible modules example
...
6: Multiple control-plane nodes and
infrastructure nodes
•
Resilient load balancing: Two CSAH nodes running HAProxy and KeepAlived
•
Dell cloud-native infrastructure: PowerEdge servers with dual NICs
•
Dell PowerSwitch: Spine-leaf fabric with VLT
Keepalived
Keepalived is an open-source project that implements routing software using the Virtual
Router Redundancy Protocol (VRRP)
...
This switchover is achieved by using VIP
...
This will be the primary
server
...
Always make external traffic paths highly available to create a complete solution
...
Deploying a resilient HAProxy requires one additional
server
...
•
Keepalived and HAProxy running on an additional server: Configure VIP on a
suitable network interface
...
6 on Dell EMC Infrastructure
Design Guide
Chapter 3: Networking Infrastructure and Configuration
Highly available load-balancing
Red Hat OpenShift Container Platform 4
...
35
OpenShift Data Foundation
...
40
CSI external storage
...
43
34
Red Hat OpenShift Container Platform 4
...
All storage within OpenShift
Container Platform 4
...
The CSI API is designed to abstract
storage use and enable storage portability
...
PVs
are internal objects against which persistent volume claims are created
...
•
PVC: An entitlement that the user creates for the specific PV
...
•
CSI driver: The software that orchestrates persistent volume provisioning and
deprovisioning on the storage array
...
Developers and operators can deploy applications and provision or deprovision persistent
storage without having any specific technical knowledge of the underlying storage
technology
...
Storage using PVCs is consumed or used in two ways: statically or dynamically
...
Static storage provisioning
With static persistent storage provisioning, an administrator preprovisions PVs for
Kubernetes tenants
...
Static provisioning is not the most
efficient method for using storage, but it might be preferred when it is necessary to restrict
users from PV provisioning
...
6 on Dell EMC Infrastructure
Design Guide
35
Chapter 4: Storage Overview
Static storage provisioning workflow
Dynamic persistent storage provisioning
Dynamic persistent storage provisioning, the most flexible provisioning method, enables
Kubernetes users to secure PV provisioning on demand
...
The following figure shows the dynamic storage provisioning workflow in this solution:
Dynamic storage provisioning workflow and benefits
36
Red Hat OpenShift Container Platform 4
...
This restriction
binds the PV to a single namespace, that of the binding project
...
PV types
OpenShift Container Platform natively supports the following PV types:
•
AWS Elastic Block Store (EBS)
•
Azure Disk
•
Azure File
•
Cinder
•
Fibre Channel (FC)—can only be assigned and attached to a node
•
GCE Persistent Disk
•
HostPath (local disk)
•
iSCSI (generic)
•
Local volume
•
NFS (generic)
•
Red Hat OpenShift Data Foundation
•
VMware vSphere
The CSI API extends the storage types that can be used within an OpenShift Container
Platform solution
...
The storage capacity can be set or requested by a pod that is launched within the
container platform
...
PV access
modes
A resource provider can determine how the PV is created and can set the storage control
parameters
...
Provider capabilities determine the PV’s access modes, while the
capabilities of each PV determine the modes which that volume supports
...
Pod claims are matched to volumes with compatible access modes based on two
matching criteria: access modes and size
...
Static persistent
storage
The use of generic NFS or generic iSCSI is functional and stable
...
Generic NFS and iSCSI do not provide the advanced storage protection
support that is available using CSI drivers
...
6 on Dell EMC Infrastructure
Design Guide
37
Chapter 4: Storage Overview
Table 7
...
OpenShift Data
Foundation provides file, block, and object storage classes that enable workloads for data
at rest, such as databases and warehouses; data in motion, automated data pipelines;
and data in action
...
As
a part of Red Hat Data Services offerings, OpenShift Data Foundation provides a
consistent experience, irrespective of the underlying infrastructure
...
Why OpenShift
Data
Foundation?
Cloud providers and system providers can offer storage for diverse workloads
...
These storage
solutions lack a service-level interface that delivers a consistent experience to users
regardless of the underlying storage technologies
...
With comprehensive support for Kubernetes, OpenShift Data
Foundation automates storage provisioning alongside the provisioning of
application resources, all of which are available from the OpenShift administrator
console
...
OpenShift Data Foundation provides
persistent block storage for databases and supports database availability needs
while providing consistency across multiple cloud platforms
...
OpenShift Data Foundation lets data scientists and those who support them deploy
and manage cloud-portable storage on demand, without requiring details about how
data is stored or how to move datasets to other platforms
...
6 on Dell EMC Infrastructure
Design Guide
Chapter 4: Storage Overview
•
OpenShift Data
Foundation
architecture
Data resilience for Kubernetes: OpenShift Data Foundation delivers containeraware backup capabilities using open standards for both persistent volume-level
backup for applications and cluster protection at the namespace level
...
Earlier releases of OpenShift Data Foundation were focused on a fully containerized Ceph
cluster that is run with an OpenShift Container Platform cluster, optimized as necessary to
provide block, file, or object storage with standard 3x replication
...
•
A single OpenShift Data Foundation cluster was deployed per OpenShift
Container Platform instance, so the storage layer could not be mutualized across
multiple clusters
...
OpenShift Data Foundation external mode overcomes these issues by allowing OpenShift
Container Platform to access a separate and independent Ceph Storage cluster, as
shown in the following figure
...
6 on Dell EMC Infrastructure
Design Guide
39
Chapter 4: Storage Overview
Alongside traditional internal-mode storage, this option gives solution architects multiple
deployment options to address their specific workload needs
...
OpenShift Data Foundation pods can either be converged onto the same
node or disaggregated on different nodes, enabling organizations to balance
OpenShift compute and storage resources as they like
...
The PowerEdge R740xd server is optimized for storage and provides an ideal
combination of performance and flexibility for software-defined storage applications
...
5-inch or 3
...
Red Hat and Intel® have collaborated to test several configurations to optimize workloads
for OpenShift Data Foundation
...
Dell storage options
For operations that are supported on Dell storage using Dell CSI drivers, see CSI Storage
Feature Support
...
The array delivers the performance that your applications demand
while consolidating block and file storage through iSCSI and Fibre Channel interfaces
...
6 on Dell EMC Infrastructure
Design Guide
Chapter 4: Storage Overview
PowerScale
storage
PowerScale is a modern, scale-out NAS solution for file storage, designed to be flexible
and reliable at any scale
...
PowerStore
5000T storage
The PowerStore 5000T standard deployment model provides organizations with all the
benefits of a unified storage platform for block, file, and NFS storage, while also enabling
flexible growth with the intelligent scale-up and scale-out capability of appliance clusters
...
PowerStore’s data-centric design with NVMe and
hardware-enabled advanced data reduction and 4:1 DRR deliver critical performance and
storage efficiency for both traditional and modern applications
...
The array provides NFS, block, and file storage for OpenShift Container Platform 4
...
Built
for multi-cloud deployment, the Unity XT 380F array provides simple and affordable
unified storage that is designed for high performance and low latency with the ability to
simultaneously run mixed-application workloads, process inline data reduction, and
provide data services without any performance impact
...
6
...
CSI external storage
Introduction
OpenShift Container Platform 4
...
This CSI API runs on the control-plane nodes to orchestrate and manage
configuration and tear-down of data-path storage operations
...
Kubernetes version 1
...
6
...
13, replacing the volume plug-in system
...
The CSI specification aims to
standardize the exposure of block and file storage systems to workloads running on
container orchestration systems such as Kubernetes
...
Red Hat OpenShift Container Platform 4
...
CSI architecture
Drivers are typically shipped as container images
...
An external CSI controller running on
infrastructure nodes has three containers: an attacher, provisioner, and driver
...
No other communication to the CSI
driver is allowed
...
The registrar registers the driver with the openshiftnode service, which then directly connects to the driver
...
19 and is available in
OpenShift Container Platform 4
...
OpenShift provides the
CSI Snapshot Controller Operator, which manages snapshot objects
...
All Dell Technologies CSI storage drivers support snapshots
...
Storage feature
support
The CSI drivers that are provided as open source by Dell Technologies enable a wide
range of storage options to be employed with OpenShift
...
For more information about deployment and operations using CSI drivers, see the Red
Hat OpenShift Container Platform 4
...
42
Red Hat OpenShift Container Platform 4
...
Protecting an
OpenShift environment is not as simple as applying a traditional data backup and
recovery solution to the container space
...
Dell
PowerProtect
Data Manager
As an innovative leader in protecting Kubernetes, Dell Technologies has evolved,
innovated, and integrated with OpenShift to address the needs of the new container
infrastructure
...
Dell PowerProtect Data Manager offers a
centralized platform to protect OpenShift workloads
...
PowerProtect Data Manager protects OpenShift Kubernetes workloads, ensuring that data
is easy to back up and restore and remains available, consistent, and durable in a
Kubernetes workload or disaster recovery situation
...
PowerProtect Data Manager provides software-defined data protection, automated
discovery, deduplication, operational agility, self-service, and IT governance for physical,
virtual, and cloud environments, enabling you to:
•
Orchestrate protection directly through an intuitive interface or empower data
owners to perform self-service backup and restore operations from their native
applications
•
Ensure compliance and meet even the strictest service level objectives
•
Leverage your existing PowerProtect appliances
For general information about PowerProtect Data Manager, see Dell EMC PowerProtect
Data Manager
...
6 on Dell EMC Infrastructure
Design Guide
43
Chapter 5: Cluster Hardware Design
Chapter 5 Cluster Hardware Design
This chapter presents the following topics:
Introduction
...
45
Cluster hardware planning
...
47
44
Red Hat OpenShift Container Platform 4
...
Usually, the platform design process ensures that the
OpenShift Container Platform 4
...
The cluster must also
be capable of being scaled out as the demand for workload handling grows
...
Many
operational factors can affect how the complexity of a container ecosystem affects
operational latencies
...
Our goal in providing this information is to help you get Day-2 operations under
way as smoothly as possible
...
Red Hat offers support for OpenShift Container Platform 4
...
Control-plane
node
requirements
Our minimum recommended control-plane node configuration is a PowerEdge R650
server with dual Intel® Gold 6330 CPUs and 192 GB RAM
...
Dell
Technologies recommends that you do not scale beyond 200 nodes, which means that
the proposed reference design is sufficient for nearly all deployments
...
Control-plane node sizing guide
Number of compute nodes
CPU cores*
Memory (GB)
25
4
16
100
8
32
200
16
64
*Does not include provisioning of at least four cores per node for infrastructure I/O handling
...
6 requires a minimum of four servers for a
three-node cluster, with each node running as both a control-plane node and a compute
node
...
You can
also expand the standard cluster with more compute nodes at any time
...
Red Hat OpenShift Container Platform 4
...
For compute nodes
that require add-in devices such as GPUs, we strongly recommend using PowerEdge
R740xd or R750 servers
...
The following table
shows the hardware configuration of the reference design:
Table 9
...
However, to expand a three-node
cluster with additional compute machines, you must first expand the cluster to a five-node cluster
with two additional compute nodes
...
Cluster configuration reference information
Quantity*
Description
Dell Technologies reference
1
Rack enclosure:
APC AR3300 NetShelter SX 42U Enclosure
APC AR3300 NetShelter SZ 42U
1
Management switch:
Dell EMC PowerSwitch S series 1 GbE switches
Dell Networking S3048-ON
2
Data switch:
Dell Networking S5248F-ON
Dell EMC PowerSwitch S series 25/40/50/100
GbE switches
or
Dell Networking S5232-ON
7-210
CSAH, Control-plane:
Dell PowerEdge R640/R650
PowerEdge R640 Rack Server
Compute nodes:
Dell PowerEdge R640/R650
PowerEdge R740xd Rack Server
or
Dell PowerEdge R740xd/R750
or
Dell PowerEdge XE2420
2-4
Power distribution unit:
APC metered rack PDU 17
...
Review the physical dimensions and
power requirements in a site survey
...
6 on Dell EMC Infrastructure
Design Guide
Chapter 5: Cluster Hardware Design
Validated hardware configuration options
Introduction
For validation test work in our laboratories, we used various server configurations for the
DTVD for OpenShift Container Platform 4
...
Dell Technologies recommends selecting
server configurations that are known to provide a satisfactory deployment experience and
to meet or exceed Day-2 operating experience expectations
...
Selecting the
server
processors
The Intel® Xeon Gold processor family provides performance, advanced reliability, and
hardware-enhanced security for demanding compute, network, and storage workloads
...
Intel® Gold
6330 processors are recommended for Dell PowerEdge R650 and R750 servers
...
When selecting a processor, consider the following criteria:
•
Processor core count: The processor core count must be sufficient to ensure
satisfactory performance of the workload operations and base services that are
running on each node
...
•
Ability to dissipate heat: During validation work with high-core-count and highTDP processors, the thermal delta (air discharge temperature minus air intake
temperature) across a server was recorded at 65°F
...
•
Compute node configurations: The design of compute nodes for use as part of
your OpenShift Container Platform cluster can use many compute node
configurations
...
The
processor architecture and core count per node selection can significantly affect the
acquisition and operating cost of the cluster that is required to run your
organization’s application workload
...
Red Hat OpenShift Container Platform 4
...
We chose a
mid-range memory configuration of 384 GB RAM to ensure that the memory for each CPU
has multiples of three banks of DIMM slots that are populated to ensure maximum
memory-access cycle speed
...
Consult OpenShift architectural guidance and consider your own observations from
running your workloads on OpenShift Container Platform 4
...
For guidance about server
memory population (location of DIMM modules in DIMM slots) and, in particular, using the
firmware setting for Performance Optimized mode, see the following Dell Technologies
Knowledge Base article: Dell EMC PowerEdge–14G Memory Population Rules updated for
certain server's configurations
...
We validated deployment and operation of
OpenShift Container Platform using magnetic storage drives (spinners), SATA SSD
drives, SAS SSD drives, and NVMe SSD drives
...
While there are no universal
guidelines, users gain insight over time into the capacities that best enable them to meet
their requirements
...
In tests, this configuration has been shown to have few adverse performance
consequences
...
When you choose to include high-I/O
bandwidth drives as part of your platform, consider your choice of network switches and
NICs so that sufficient network I/O is available to support high-speed, low-latency drives:
•
Hard drives: These drives have lower throughput per drive
...
•
SATA/SAS SSD drives: These drives have high I/O capability
...
SAS
SSDs operate at up to 10 times the I/O level of a spinning hard drive
...
•
NVMe SSD drives: These drives have high I/O capability, up to three times the I/O
rate of SAS SSDs
...
The following table provides information about selecting NICs to ensure adequate I/O
bandwidth and to take advantage of available disk I/O bandwidth:
48
Red Hat OpenShift Container Platform 4
...
NIC and Storage selection to optimize I/O bandwidth
NIC selection
Compute node storage device type
2 x 25 GbE
Spinning magnetic media (hard drive)
2 x 25 GbE or 4 x 25 GbE
SATA or SAS SSD drives
4 x 25 GbE or 2 x 100 GbE
NVMe SSD drives
True network HA fail-safe design demands that each NIC is duplicated, permitting a pair
of ports to be split across two physically separated switches
...
This switch is cost-effective for a compact cluster
...
The PowerSwitch S5232F-ON provides 32 x 100 GbE ports
...
Low latency in
an NFV
environment
NFV-centric data centers require low latency in all aspects of container ecosystem design
for application deployment
...
We strongly
recommend using only NVMe drives, NFV-centric versions of Intel® CPUs, and, at a
minimum, the PowerSwitch S5232F-ON switch
...
Power
configuration
Dell Technologies strongly recommends that all servers are equipped with redundant
power supplies and that power cabling provides redundant power to the servers
...
For consistency,
connect all right-most power supply units (PSUs) to a right-side PDU and all left-most
PSUs to a left-side PDU
...
Each PDU must have
an independent connection to the data center power bus
...
6 on Dell EMC Infrastructure
Design Guide
49
Chapter 5: Cluster Hardware Design
PSU to PDU power template
50
Red Hat OpenShift Container Platform 4
...
52
Enterprise applications
...
55
Data analytics and artificial intelligence
...
6 on Dell EMC Infrastructure
Design Guide
51
Chapter 6: Use Cases
Overview
This chapter describes how an OpenShift Container Platform 4
...
The
examples in this chapter include enterprise application development and deployment,
telecommunications service provider operations, and data analytics and artificial
intelligence
...
These applications offer
the agility, speed, and composability of microservices architecture
...
•
Large enterprise deployments might cater to many internal and external partners
and customers
...
Security and isolation are required between
microservices
...
Enterprises must deploy a single cluster or multiple highly scalable large clusters in each
data center
...
One of the
key decisions we made regarding multisite OpenShift Container Platform 4
...
A major advantage of this
model of deployment is disaster avoidance
...
The following figure shows a multisite deployment of this solution:
52
Red Hat OpenShift Container Platform 4
...
6
•
Load balancer: Global traffic manager (GTM) and local traffic manager (LTM)
•
Data center hardware infrastructure
Access control has implications for what multitenancy means throughout the
infrastructure
...
For example:
•
A provider administrator must be able to see usage and metering information for the
entire infrastructure
...
•
Tenant users require access only to assets and resources that they are permitted to
manage
...
6 can be linked to
your Microsoft Active Directory identity management environment or other supported
identity managers
...
The following table shows the roles that OpenShift Container Platform 4
...
6 on Dell EMC Infrastructure
Design Guide
53
Chapter 6: Use Cases
Table 12
...
6
Role
Description
admin
Project manager
basic-user
User who can get information about projects and users
cluster-admin
A superuser who can perform any action in any project
cluster-status
User who can get cluster status information
edit
User who can modify objects in a project
self-provisioner
User who can create their own projects
view
User who can see most objects in a project
cluster-reader
User who can read, but not view, objects in the cluster
Security and
isolation
OpenShift Container Platform 4
...
The project manager must have
the administrative privilege to be able to see any other project in the cluster
...
OpenShift Container Platform 4
...
For more information
about obtaining cluster resource consumption to drive usage billing through third-party
application software, see the following Red Hat documentation:
•
About cluster monitoring
•
Examples of using metering
•
About cluster logging and OpenShift Container Platform
The cluster monitoring operator controls the monitoring components that are deployed,
and the Prometheus operator controls Prometheus and Alert manager instances
...
6 on Dell EMC Infrastructure
Design Guide
Chapter 6: Use Cases
•
Kubernetes scheduler
•
Metering
•
OpenShift apiserver
•
OpenShift controller manager
Networking
Introduction
A typical telecommunications (telecom) company sells telecom-oriented applications as a
service to its consumers
...
These functions include:
•
Content delivery network (CDN)
•
Edge infrastructure and towers of power
•
NFV management and operations (NFV-MANO)
•
Software-defined networking (SDN) and SD-WAN management
•
Radio access networks (RAN) and 5G, and their component service infrastructures
•
Multiaccess Edge Computing (MEC)
•
Core network and 5G Next Generation Core (NGC)
This use case identifies some key design factors for a telco container platform
...
High-quality video delivery over
public networks requires a CDN
...
CDN
virtualization permits simple provisioning and sharing of resources with other telecom
services, simplifying operations and avoiding costly dedicated infrastructure
...
6 on Dell EMC Infrastructure
Design Guide
55
Chapter 6: Use Cases
The following figure shows a virtual CDN (vCDN):
Virtual content delivery network (vCDN)
vCDN platform
requirements
A vCDN stack requires the following principal capabilities:
•
Large application storage space to store video and other files
•
High-speed and low-latency network options to serve the content
•
Rapid ramp-up of on-demand processing capacity
OpenShift Container Platform 4
...
These drivers can be integrated into your OpenShift Container Platform deployment
using the new CSI plug-ins
...
•
OpenShift Data Foundation (based on Ceph) is supported as part of OpenShift
Container Platform 4
...
•
Multus CNI plug-in support by which additional networks can be added to each
container so that the container can meet capacity needs on targeted networks
...
Red Hat
provides an SR-IOV Operator over OperatorHub, enabling administrators to
manage virtual functions on nodes through Kubernetes CRDs
...
6 as a Technical Preview
feature
...
Red Hat OpenShift Container Platform 4
...
•
For low-latency workloads, you can configure compute nodes running Red Hat
CoreOS to use the real-time kernel, which provides consistent low-latency
performance
...
•
Small footprint clusters (three-node clusters) can also be deployed at the edge for
production environments
...
•
Telco applications generally use huge pages
...
•
OpenShift Container Platform 4
...
Container ecosystem clusters for large deployments can span across multiple racks
...
The deployment of large clusters requires significant modification of the Ansible
playbooks that were generated to facilitate large-scale deployment
...
All these disciplines benefit from running in containerized environments
...
For simplicity, we use the term “data analytics as a service” (DAaaS) for analytics and AI
that are operated and instantiated in a containerized environment
...
This delivery model makes various
tools available for data analytics and can be configured to efficiently process and analyze
huge quantities of heterogeneous data from shared data stores
...
At a high level, the life cycle consists of these phases:
•
Data acquisition and preparation—Ensures that the input data is complete and of
a high quality
...
Red Hat OpenShift Container Platform 4
...
Data scientists and engineers are primarily responsible for developing modeling methods
that ensure that the selected outcome continues to provide the highest prediction
accuracy
...
OpenShift Container Platform provides all these benefits
...
OpenShift Container Platform also
accelerates the roll-out of analytics applications to departments as needed
...
▪
Eliminating dependency on IT to provision infrastructure for iterative, computeintensive ML modeling tasks
...
•
Streamline the development and operations of intelligent applications:
Extending OpenShift DevOps automation capabilities to the ML life cycle enables
collaboration between data scientists, software developers, and IT operations so
that ML models can be quickly integrated into the development of intelligent
applications
...
MLPerf benchmarks are an independent valuation of
performance for various parts of the machine learning ecosystem, including both the cloud
Red Hat OpenShift Container Platform 4
...
The MLPerf training and inference benchmarks were
run on top of OpenShift and compared to NVIDIA’s MLPerf benchmark results
...
The
results indicated that the addition of the OpenShift platform did not hamper the
performance of intensive ML applications and demonstrated that OpenShift provides
valuable benefits for running ML applications in production environments
...
Kubeflow is an open-source Kubernetes-native platform for ML workloads that enables
service providers to accelerate their ML/DL projects
...
Kubeflow bundles popular ML/DL frameworks such as TensorFlow, MXNet, PyTorch, and
Katib with a single deployment binary file
...
For more information, see the Machine Learning Using Red Hat OpenShift Container
Platform (this white paper is based on the OpenShift Container Platform 4
...
For more information, see Kubeflow: The Machine Learning Toolkit for Kubernetes
...
Apache Spark, a unified analytics
engine for big data and ML, is one of the largest open-source projects in data processing
...
OpenShift orchestrates the
creation, placement, and life cycle management of those Spark processes across a
cluster of servers by using container virtualization to host the processes
...
SQL Server big
data clusters on
OpenShift
Another example of big data analytics being run on OpenShift is the Dell Technologies
solution for Microsoft SQL Server 2019 Big Data Clusters
...
These components run side
by side to enable you to read, write, and process big data so that you can easily combine
and analyze your high-value relational data with high-volume big data
...
For more information, see the Microsoft SQL Server 2019 Big Data Clusters White Paper
on the Dell Technologies Info Hub for SQL Server
...
6 on Dell EMC Infrastructure
Design Guide
59
Chapter 7: References
Chapter 7 References
This chapter presents the following topics:
Dell Technologies documentation
...
61
Other resources
...
6 on Dell EMC Infrastructure
Design Guide
Chapter 7: References
Dell Technologies documentation
The following Dell Technologies documentation provides additional information
...
If you do not have access to a
document, contact your Dell Technologies representative
...
6 on Dell EMC Infrastructure
Implementation Guide
•
Dell EMC PowerEdge R640 Technical Guide
•
Dell EMC PowerEdge R740 and R740xd Technical Guide
•
Dell EMC PowerEdge R650 Technical Guide
•
Dell EMC PowerEdge R750 Technical Guide
•
Machine Learning Using the Dell EMC Ready Architecture for Red Hat OpenShift
Container Platform (this document is based on OpenShift Container Platform 4
...
•
Dell EMC Unity: Best Practices Guide
•
Dell Technologies Solutions Info Hub for Communication Service Providers
•
Dell EMC Ready Stack Converged Infrastructure
•
Dell EMC PowerProtect Data Manager Protecting OpenShift Workloads
•
Dell EMC PowerMax Storage
•
Dell EMC PowerScale Storage
•
Dell EMC PowerStore Storage
•
Dell EMC Unity XT Storage
•
Dell EMC PowerFlex for OpenShift Installation and Configuration Guide
•
Dell Technologies Container Storage Modules and CSI Drivers GitHub
Note: For links to additional relevant documentation, see the Red Hat OpenShift Container
Platform section of the Dell Technologies Info Hub for Containers
...
6 on Dell EMC Infrastructure
Design Guide
61
Chapter 7: References
•
Planning your environment according to object maximums
Other resources
The following resources provide additional information:
62
•
Intel® Xeon Gold Processors
•
Kubeflow: The Machine Learning Toolkit for Kubernetes
•
Prometheus: From metrics to insight
•
Operating etcd clusters for Kubernetes
•
NVIDIA Tesla GPU Documentation
Red Hat OpenShift Container Platform 4
...
64
Dell PowerEdge R640 node BOM
...
65
Dell PowerEdge R650 BOM
...
67
Dell PowerStore 1000T BOM
...
69
Dell PowerMax BOM
...
70
Red Hat OpenShift Container Platform 4
...
The memory, CPU, NIC, and drive configurations that are shown in
the tables are preferred but not mandated
...
Dell PowerEdge R640 node BOM
The following table lists the key recommended parts per node
...
Table 13
...
5-inch-inch chassis with up to 10 hard drives, 8 NVMe drives, and 3
PCIe slots, 2 CPU only
2
338-BTSI
Intel Xeon Gold 6238 2
...
4GT/s, 30
...
6 m) power cable, North
America
Red Hat OpenShift Container Platform 4
...
6 TB, NVMe, Mixed Use Express Flash, 2
...
2,
P4610 with Carrier
2
400-AZQO
800 GB SSD SAS Mix Use 12Gbps512e 2
...
2 240G, R1, LP1
Dell PowerEdge R740xd node BOM
The following table shows the PowerEdge Server R740xd baseline configurations that are
used in this design for OpenShift Container Platform 4
...
Table 14
...
5 -inch hard drives including 12 NVME drives, 2
CPU configuration
1
338-BTSI
Intel Xeon Gold 6238 2
...
4GT/s, 30
...
6 on Dell EMC Infrastructure
Design Guide
65
Appendix A: Hardware Configuration
Qty
SKU
Description
1
492-BBDH
C13 to C14, PDU Style, 12 AMP, 2 ft (0
...
2 240G, R1, FH
Select one of the following rows:
1 to 24
Check part at
time of ordering
800 GB, 1
...
84 TB SSD SAS mixed use 12 Gbps 512e 2
...
6 TB, 3
...
4 TB, NVMe, mixed use express flash, 2
...
2, P4610 with carrier, CK
Dell PowerEdge R650 BOM
The following table shows the PowerEdge Server R650 baseline configurations that are
used in the Dell design for OpenShift Container Platform 4
...
Table 15
...
5, 10x2
...
5 Chipset NVMe RAID Config,
2CPU
1
405-AAZE
PERC H755N Front
1
330-BBRP
Riser C0-2, 3x16 LP, HL
1
403-BCMB
BOSS-S2 Cntrl + 2 M
...
6 TB, NVMe, 2
...
0
1
540-BCMQ
Mlnx ContX-5 DP 25 Gb SFP
Adpt, LP
Red Hat OpenShift Container Platform 4
...
6:
Table 16
...
6 TB, NVMe, 2
...
5" NVMe RAID
1
330-BBRW
Riser Config2, Full Length,4x16,2x8 slots
1
540-BCNM
Mlnx ContX-5 DP 25Gb SFP Adpt, FH
1
403-BCMB
BOSS-S2 Cntrl + 2 M
...
0
Dell PowerStore 1000T BOM
The following table shows the PowerEdge Server 1000T baseline configurations that are
used in this design for OpenShift Container Platform 4
...
Table 17
...
FLD INST
1
343-BBMR
BASE UNIT CONFIG KIT
1
370-AEZP
384 GB Appliance DIMM 192 GB Per Node
8
400-BGGI
P1 25 X 2
...
92 TB
1
528-BTZK
PowerStore Base SW
1
406-BBQI
10 GBE OPTICAL 4 PORT CARD PAIR
1
565-BBJP
16 GB FC 4 PORT IO MODULE PAIR
Red Hat OpenShift Container Platform 4
...
6 on Dell EMC Infrastructure
Design Guide
Appendix A: Hardware Configuration
Dell Unity 380F BOM
The following table shows the Dell Unity 380F baseline configurations that are used in this
design for OpenShift Container Platform 4
...
Dell Unity 380F BOM
Qty
SKU
Description
8
D4F-2SFXL2-1920
D4F 1
...
5 SSD
1
D4ODPEKITAF
UNITY 380F DPE INSTALL KIT
2
C13-PWR-12
2 C13 CORDS NEMA 5-15 125 V 10A - NON DPE
1
D4BD6C25FAFLL
UNITY 380F DPE 25 X 2
...
6:
Table 19
...
92 TB
1
EH-PCBL3DHR
PWR CBL HBL-RSTOL 3D
2
EH-ACON3P-50
ADPTR AC 3PH 50A W3-4IN CONDUIT ADPTR
1
EH-VBBASE-KIT
POWERMAX 2000 PB BASE INSTALL KIT
2
EH-DE24
POWERMAX 2000 DIRECT 24 SLT DR ENCL
1
EHX-BEDIR
POWERMAX 2000 PRO DIR
1
EH-1024BASE
POWERMAX 2000 BASE 1024 GB
1
EH-FE00800T
POWERMAX 2000 8MM 10GIGE
Qty
Red Hat OpenShift Container Platform 4
...
92 TB SPARE
2
EH-1600MOD
POWERMAX 2000 FLASH MOD 1600
8
E-GE-ISCSI
VMAX VG GIGE ISCSI PORT TRACKING
MODEL
3000
E-DRR
POWERMAX 2/8K DATA REDUCTION
RESERVATION
1
E-Q118E
ELM TRACKING MODEL
13
E-OPROVISION
OPROVISION FACTOR TRACKING MODEL
1
WKPROFILE-BAL
VMAX VG WORKPROFILE BALANCED
1
EH-MGMT
EMBEDDED MANAGEMENT POWERMAX 2000
TRK
1
M-PSM-HW-020
PROSUPPORT 4HR/MC HARDWARE SUPPORT
11
458-002-223
POWERMAX PRO SUITE OS 1 TB=CC
1
M-PSM-SW-020
PROSUPPORT 4HR/MC SOFTWARE SUPPORT
1
450-001-644
POWERMAX PRO SUITE=IC
1
M-PSM-SW-020
PROSUPPORT 4HR/MC SOFTWARE SUPPORT
11
450-001-645
POWERMAX PRO SUITE 1 TB=CC
1
M-PSM-SW-020
PROSUPPORT 4HR/MC SOFTWARE SUPPORT
1
PS-PD-PMAX2DP
PD FOR POWERMAX 2000
1
PSINST-ESRS
ZERO DOLLAR ESRS INSTALL
1
CEVALPAKPMAXSADM
POWERMAX/VMAX AF STORAGE ADMIN
VALUEPAK
Qty
OpenShift Data Foundation data node configurations
Three workload-optimized configurations for OpenShift Data Foundation external data
nodes have been developed for consumption based on testing that Red Hat and Intel
conducted
...
Each configuration is available in a Base configuration and
a Plus configuration
...
Each configuration
features the most appropriate Intel® Xeon Scalable processor, Intel® SSDs, and Intel®
networking products
...
Core-for-core, these processors offer
industry-leading performance on popular databases, HPC workloads, and AI
...
6 on Dell EMC Infrastructure
Design Guide
Appendix A: Hardware Configuration
configurations will take advantage of performance enhancements and increase business
value across a wide range of workloads
...
Red Hat OpenShift Container Platform 4