55

Service Fabric placement constraints and cluster planning – virtual clusters

 5 years ago
source link: https://www.tuicool.com/articles/hit/jAjyYfB
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Introduction

In this article we see how to achieve the service placement using placement constraints. I have written this post as a continuation of this previous article . I recommend to read it, especially if you’re new to Service Fabric (SF).

As per the article, we should place WFE services in certain set of nodes which are exposed to LB and other internal services in a different set of nodes which are not exposed to LB and may have access to the data layer.

In fact what I have tried to achieve is a typical infrastructure setup with DMZ. The difference is one single SF cluster holds the DMZ and non DMZ. 

We can achieve this with placement constraints in SF. In the simplest form placement constraints work based on the properties we set to the nodes. Node properties are key value pairs used to tag nodes. Later through the application we can instruct SF to place certain services in certain nodes which satisfy the placement constraint logic.

You can configure the node properties in Azure portal under the node types. In the on premise setup we can edit ClusterConfig.json to add node properties . Like any configuration, placement constraints can also be parameterized in the ApplicationManifest.xml using corresponding parameters xml file. This article describes it very clearly. 

Setup – Virtual Clusters

In a sample setup with 6 nodes and FD:UD = 6:6 the DMZ and non DMZ setup looks like the below setup.

Zni6RvR.png!web

Node properties make the DMZ and infrastructure setup will give the implementation level separation. In our case we place the DMZ and non DMZ in two different subnets and configured a software firewall in between both subnets. Also the nodes in DMZ subnet cannot talk to the databases.

WFE services are placed in the DMZ (red box) and internal services are placed in non DMZ (yellow box).

When it comes to scaling each zone will act as a virtual cluster.

Constraints and tips to note.

This is the section of the article where we’re going to get into some details of placements. I have provided simple rules that will help to achieve optimum setup and troubleshooting with experience in the cluster.

When setting the cluster we have to specify the Update Domains (UD) and Fault Domains (FD). In the simplest form UD:FD is a 1:1 setup. It serves majority of the purpose,  though we can have any number of nodes in the cluster, when we specify placement constraints the service replicas have limited number of nodes available to them.

Instance / Replica : The term instance is used to refer the stateless service copies and replica is used to refer the stateful service copies but in this article I have used the term replica to refer both.

First let’s look how SF places the services when there’s no placement constraints defined. The default placement approach SF has is known as adaptive approach. It is mix of two approaches known as maximum difference  and  quorum safe. 

  • Maximum difference is a highly safe placement approach where any replica of a single partition will not be placed in same FD/UD
  • Quorum safe approach is minimal safety mode where it happens only when the below IF condition is true. Here SF tries to be economical of the nodes.

In the adaptive mode SF chooses the right approach based on this below formula.

rs: replica size, fd : fault domain, ud: update domain n: number of nodes

if ( rs % fd == 0 && rs % ud == 0 && n <= (fd * ud) )
        return "quorum safe"
else
       return "maximum difference"

Choosing the above any approach would not yield the successful placement. Because this a decision of placement strategy, once the decision is made if there’s no enough nodes to place the services SF will complain either as an error / warning depending on the situation.

When we define the cluster setup and planning with placement constraints, it is very important and little tricky to think on the capacity planning because we may end up in wrong configurations.

I sort of summarized the placement constriant

Placement constraints are static

Optionally we can inject some software firewall in between the WFE nodes and the nodes host internal services.

Summary

Rule #1: Any replica belongs to a single partition cannot more than the number of fault domains in SF cluster. ie:

  1. Regardless of how many

report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK