Notes on Azure Fundamentals

This long-scrolling post is nothing but the notes I took while learning AZ Fundamentals. Most of it is my own writing, with a few details directly copied from the Azure Documentation. Images are almost always taken from the Azure website, with a few being my screenshots.

Since the whole content is already freely available on Microsoft website, I believe there is no copyright issue in this. If there is, please alert me.

Because, Ignorantia juris non excusat!, or something like that ;-)

Ref: https://aka.ms/azfunpath

Those who plan to take the AZ 900 exam should at any cost read the Exam Skills Outline document. The importance of this cannot be stressed more.

Azure Fundamentals | Part-1

Tour of Azure Services

Azure Services into Broad Categories

1. Compute Services
   • VMs, Containers, Serverless Computing including Microservices
2. Cloud Storage
   • Disks attached to VMs, structural formats like fileshares and databases
3. Networking
   • Configure and control your networking to send traffic in and out of Azure, like private networks
4. App Hosting
   • Run web apps, marketplace which gives ready solutions
5. Artificial Intelligence
   • ML & prebuilt cognitive services, big data analytics
6. Internet of Things
   • Integrate things with networks
7. Integration
   • Logic apps, orchestration, etc.
8. Security
   • Integrated into every aspect of Azure Services; control who has access to what, etc.

Check this Image below to see Azure Services on a high level.

Most Commonly Used Service Categories

• Compute
• Networking
• Storage
• Mobile
• Databases
• Web
• IoT
• Big data
• AI
• DevOps

What are virtual machine scale sets? Refer this link

Azure virtual machine scale sets let you create and manage a group of load balanced VMs. The number of VM instances can automatically increase or decrease in response to demand or a defined schedule. Scale sets provide high availability to your applications, and allow you to centrally manage, configure, and update a large number of VMs. With virtual machine scale sets, you can build large-scale services for areas such as compute, big data, and container workloads.

Azure Compute Services

• VMs are also called as Compute Nodes
• Azure VMs - provides Windows and Linux VMs
• Azure VM Scale Sets - create and manage a group of load balanced VMs, set to automatically increase or decrease the no. of VMs in response to demand or schedule
• Azure Kubernetes Service - manage containerized VMs
• Azure Service Fabric - manage microservice based application setups; runs anywhere on other clouds, on Linux or on-prem
• Azure Batch - run parallel and HPC batch jobs scheduled on multiple compute nodes
• Azure Container Instances - run containerized applications without provisioning servers or VMs
• Azure Functions - This needs a special mention. So I wrote about it below.

Serverless Computing - The coming-together of Azure Functions, Logic Apps and Event Grid

This part needs a special mention because this is just one feature out of the many coming out the concept of Serverless Computing or Serverless Application.

• Serverless computing is essentially a framework within which the client just worries about writing code so their application work seamlessly, without having to spend their time on the number of VMs they need, the amount of storage they need, the kind of networking they need, the level of security they need, etc. All these things will be automatically scaled up and down based on the app’s requirement and other parameters.
• I just watched a simple 20 minutes YouTube video on this subject and the use case shown here is really stupid simple.
• Serverless Computing on Azure works with 3 most critical services provided by the platform.
  • Azure Functions
  • Azure Logic Apps
  • Azure Event Grid
• The video above is essential to understand how these 3 features work in harmony to ensure a serverless app runs seamlessly, just taking care of some manual work which otherwise would have taken number of hours, or even days and number of employees workload.
• But to put it simply, here is the use case shown in the above video.
  • It is called Employee Onboarding process. Now we all know how boring and repetitive this is and how much time the HR staff end up spending on. Azure Functions, Logic Apps and Event Grid functionalities will take care of this whole process, and produce results in perhaps 1-2% time and with 1-2% of the efforts.
• I may sound exaggerating, but I am honestly impressed when I saw the demo in the video.
• The only issue is that the staff who create and maintain such workflows should be really efficient, and would need a positive encouraging support from the bosses.

Azure Networking Services

NETWORKING - Linking compute resources and providing access to applications is the key function of Azure networking. (Copied from the Azure Fundamentals documentation web page)

The work Networking is surely one of the most enigmatic things in Computer Science and Technology. No matter how much is said and written about it, it is still one of the most fascinating areas.

Particularly when networking is being talked about in Data Center and Cloud related subjects, it’s attractiveness gets amp’ed up quite hardly, because it surely needs a good grasp of quite a few subjects, including mathematics, geography, weather science, electronics, engineering, and what not. Of course, that’s why people work in teams with members of their own specialities.

So, not much of notes can be written on this subject, because it usually takes months to years to learn this subject anyway.

As summary, Azure can do pretty anything and everything you can think of under the title “Networking”, mostly automatically.

Azure Storage

Azure provides 4 types of Storage Services.

1. Blob Storage - for very large data, such as videos
2. File Storage - usual file server
3. Queue Storage - for queuing and delivering messages between applications
4. Table Storage - for unstructured data, usually a NoSQL store, independent of schema

All storage services come with the following features:

• High availability and Redundancy
• Automatic Encryption and RBAC
• Scalability
• Maintenance by Microsoft
• Most of these storage services can also be configured to be accessible from mobile phones, made for all flavors - iOS, Windows and Android, along with Push notifications.

And More ….

Azure Services include a lot of other services that, similar to networking, cannot be explained in a single post like this.

For instance, Azure Database Services offer a wide range of services that include use cases with almost all flavors of database types, such as MySQL, NoSQL, PostgreSQL, CosmosDB, MariaDB, etc.

Azure Web Services include all things web applications - notification hubs, API management, real time web functionalities, etc.

Azure IoT, Big data, AI and DevOps include a vast number of services that enable and enhance daily activities in the modern technology-driven world, right from letting a vacuum cleaner determine whether its job for the day is complete or not depending on the dust it can sense on the floor, up to visual analytics of a poacher in Amazon forest who attempted to poach a rhino and got caught, first in the sensor-based camera in the forest and then by the police before they could run more than 10 km in the forest.

Take Away, truly! Microsoft Learn Sandbox

Sandbox is nothing but Azure resources made available for the learner to practice and get some hands-on feel on the Azure Services. This is free of cost and time-bound resource. It is really helpful to get the hang of how managing your cloud on Azure portal feels like.

Also, you would be able to clearly understand the workflow. Once it is clear in our mind what we want to create, depending on that object, we can see what the workflow is. For example, before you do anything, you will have create a Resource Group. Anything and everything you create begins with this step. And then the individual workflows will begin. You will go to marketplace and choose a ready-made service, or will create something from the scratch, either by creating a new VM, or a new DB, etc.

Another very crucial critical skill one learn using the Sandbox is to practice the Azure CLI. This is surely an added feather in the hat, once learned sufficiently. You almost never have to log into the Azure portal and everything can be managed just from the comfort of your terminal window. Thanks to Microsoft’s open collaboration with Linux, you can manage your Azure services both from Windows and Linux OS command line tools.

Azure Fundamentals | Part-2

Azure Architecture and Service Guarantees

Azure Regions and Data Centers

Azure has established its data centers in a large number of geographical regions, spanning almost all the continents. It has a lot of data centers in its motherland USA, with a few dedicated, network-isolated, specialized data centers for US government departments. Outside, they have setups in the EU, Australia, New Zealand, UAE, Qatar, Saudi Arabia, Israel, India, Chine and the far east.

This vast set of regional setups ensure that the clients who depend on Azure Services can always choose the closest possible region to ensure faster reach, and multiple data centers in most regions ensure the business continuity.

Some regions in the far east, for example China, are not maintained by Microsoft, but by a Chinese company called 21Vianet.

But in Azure technical terms, Regions are different from Geographies and Availability Zones.

Azure divides Geographies into Regions for various purposes, such as US West, Australia Central, South India, etc.

• One of the ways they divided the geographies is based on the compliance requirement, therefore the regions in these geographies and the data centers in those regions can be setup based on the same compliance and regulatory requirements.
• Another reason is customer-centric. Clients who wish to address the data residency or sovereignty requirements can also benefit from this geographies concept.
• One more critical purpose is to ensure a whole region failure does not impact those business who focus on business continuity and high availability more than anything else, and thereby become fault-tolerant. For example, international banks.

Geographies are divided into Americas, Europe, Asia Pacific, Middle East and Africa

Finally, Availability Zones. These are separate data centers within a particular region. Sometimes, one zone can have multiple data centers.

Availability Zones are primarily for VMs, managed disks, load balancers, and SQL databases. Azure services that support Availability Zones fall into two categories:

• Zonal services – you pin the resource to a specific zone (for example, virtual machines, managed disks, IP addresses)
• Zone-redundant services – platform replicates automatically across zones (for example, zone-redundant storage, SQL Database).

So,

• One or more data centers in an Availability Zone …
• One or more Availability Zones in a Region …
• One or more Regions or Region Pairs in a Geography !!!

In addition, there are also Region Pairs. A region pair consists of two regions physically at least 300 miles away from each other, with the second one working as a fail-over the moment the first one is down due to any major cause, such as a natural disaster.

The Brave New World of Data

Data is identity. Data is business. Data is careers. Data is the job. Data is everything. Data is everywhere.

Image Source: https://draup.com/talent/whitepapers/us-ai-and-big-data-talent-assessment/

Every transaction we make either depends on available data or gives rise to new data. Doesn’t matter if it is logical or emotional, monetary or not, tangible or not, recorded or not. Both visible and invisible data are all-pervasive in human lives.

Among such ubiquitous material, what makes money is held in great respect and called “Data”, while the rest goes into irretrievable history.

Every business wants to make money. Every business that makes money needs and produces more such data.

At one time, money or gold used to be so precious. Losing them would have dire consequences. Data is the new entrant into the elite. These days, losing data has consequences much direr than when, say, money is lost.

Then, we needed to secure money, like the most precious possession in our life. Now, we need to secure and handle data with just as much care if not more, because one wrong decision can have unimaginable impacts on individuals, corporates, or even economies. Such is the regulatory landscape around data these days.

That brings us to the already well established data security domain. This is so vast a topic that despite the thousands of volumes and series of books written on this subject, it still remains an enigma for most people. Perhaps, like any other technical subject appears to a non-technical person.

Data and its security creates work for both kinds of people, good and bad. Business starts minding its business. Bad will come in and try to steal the data. Good will try to stop the mishap, or will learn from failing to stop. These days, even a startup takes data security very seriously. But would it exist without someone who manages the data.

That calls for a precursor - Data Management and Governance. The only way to ensure that data security guys will have a bit easier job either in stopping a breach, or investigating a breach, is to commit to a solid platform, that is data management. If data management is not up to the standard, securing it becomes almost a theoretical subject for C-suite discussions, resulting in no tangible action.

The challenges caused by data not so well managed from its inception can lead to nightmares, they are often ruthless and painful.

We already have an idea how vast these fields could be based on the severity that is explained so far. But imagine the situation that you are challenged on daily basis with questions related to data management, data security, data privacy/protection, data on cloud, data shared with a service provider, etc. This complex and challenging are our daily routines becoming now, and this is just the beginning.

• The notorious world of AI/ML combo
• The robotics that are already entering the hospitality and medical industries
• The scary world of crypto currencies and blockchains (or the digital ledgers)
• The ongoing noise around digitalization of banks and fintech
• The ever rising Internet of Things (IoT) products and services
• The unstoppable (apparently) world of social media

And more domains like these are just starting to look straight into our eyes, and we are yet to learn to look them eye to eye. These fields are producing insurmountable volumes of data and are expected to ooze exponentially as each minute passes. Because data is produced in less than a minute these days with the data processing and storing systems we have currently. The future is always interesting.

Present is even more exciting as we can witness how all this is shaping up, like in a time lapse video of a flower blossoming, or a seed sprouting.

There are numerous documentaries and news echoing around the data challenges and some of them just brilliant and thought-provoking.

1. Snowden
2. Don’t f**k with cats
3. The Social Dilemma
4. Facebook & Cambridge Analytica Scandal
5. Schrems II & the Privacy Shield

One useful take away

It is no longer sufficient that you develop skills in a single narrow specialized field. To become efficient in daily work, we need to leverage on knowledge gained in all these areas because none of these work in silos.

One of these days, I had the opportunity to read a draft publication and the beginning statement is so true and stimulating, that it rightfully demands all our attention.

With the explosion of the digital universe, it is becoming increasingly important to understand how organizational decision making (i.e., the business-oriented perspective) is intertwined with an understanding of enterprise data assets (i.e., the data-oriented perspective). - Vijay Khatri

What it means to get schooled?

Should life flow seamlessly or happen in bits and pieces, like accidents?

Do you know that even when you look at a tree and say, “That is an oak tree”, or “that is a banyan tree”, the naming of the tree, which is botanical knowledge, has so conditioned your mind that the word comes between you and actually seeing the tree? To come in contact with the tree you have to put your hand on it and the word will not help you to touch it. – Jiddu Krishnamurti, Freedom from the Known

Image Source: g-stockstudio/Shutterstock

Life happens in phases for most. Especially for those who get schooled. Schooled not only by schools, but on daily and constant basis, by parents and people in the family, by neighbors, by society, by friends and their families, by news and media opinions, by ancient and current “authoritative” texts, the so called realized beings, etc.

For these, life happens in phases. They begin with a phase where they only enjoy but not work at all. This is their early childhood times. This is the period when they begin to go through the process of getting conditioned, polished, programmed, disciplined, and so on and on. They get to see the world suddenly in duality — light vs dark, morning vs night, sun vs moon, mom vs dad, human vs animal, empty stomach vs filled one, etc. and finally reaching up to good vs bad as well as slowly but in smaller doses, ethics and morals. And a special mention for the echoing instruction from parents, “Behave!”

They are then slowly introduced to enjoy more and work less in their early school days. These are the times they simple enjoy a smile, a rainy day for no reason. Well, that’s important, for no reason. They enjoy playing in a water tank large enough to hold two small kids. They enjoy watching touch-me-not plants. They enjoy nights through star gazing and sleeping on the terrace or in the open sky. They enjoy watching other animals with no expectations. They enjoy creating things with clay kind of simple and natural things.

They don’t need anything extraordinary, any sophisticated items made with supreme technology, to walk in pursuit of happiness. Because they do not know they need to pursue it, they think being happy is all that there is to be. Their inherent happiness instead brings ample opportunities for them to just be in the moment.

Eventually, they reach up to balancing their work and enjoyment. These are higher school and college days. Now career becomes a topic of discussion wherever they go, mostly initiated by the person they ended up meeting. Now they are told to work more and enjoy less.

The moment they are introduced to another human, the first thing that happens to them is, they get advised!

This goes on until their early marriage days. Once the life partner is not a novelty in their life anymore, stability and security take the dominant part of their life. They only work in this phase, but not enjoy. They usually find occasional opportunities to have their time out, maybe trips, long holidays, family outings, movies, social media, streaming platforms or other entertainment. It’s their effort to balance work and enjoyment, with a clarity already in their minds that they will anyway fail at it utterly — balancing. So, they fail to enjoy because their minds are now filled vessels. They always carry a baggage without noticing that they could actually take a break and keep the baggage aside for awhile. And this is the dullest and perhaps longest phase of life.

What you do to suppress the effects of your work is not enjoying.

And then the last phase of life is obviously, fully, and wholly dependent on how life is spent so far. Here they are rightful to go back to times when they only enjoy but no work. But rights don’t guarantee anything. Their physical bodies should support. And many other such dependencies…. How their children settled in life, how much they saved for their last phase, how many more responsibilities they feel they fulfilled and how many are left incomplete, etc. They still struggle to figure out, where freedom is, and when their time is to be free. They keep needing it but hardly find it. It starts becoming hard to exist.

To be or not to be.

Unfortunately even at this stage, most people prefer to imagine the next life fully based on the life they lived so far. Instead, why can’t they simply think they are going back to where they came from. But their schooled minds won’t give them that space for keeping things and thoughts simple anymore.

Finally, the judgement day comes. And all shall pass.

Doesn’t this sound a very pessimistic and tragic depiction of circle of life? I feel so but can’t find a better way.

Now, relax your muscles and think of a rare class of humans. They are not schooled. These are the ones whose parents chose and strive towards staying away from programmed thinking. They may not be able to achieve it themselves. But they decide to create an ambiance for their kids to remain untouched by society’s programming. To the best possible extent, that is! They are taught to learn and apply things on their own on daily basis. They are taught not to enjoy what they get to do. But to do only what they truly enjoy.

Have you ever seen their faces? There’s usually a shine on their faces. Their smiles feel like coming from their hearts, but not from their lips. Even their eyes seem like smiling. They don’t worry about failing. Because the clarity of thought that they develop by doing only what they like to do always keeps them positive about failing. These are the ones who become artists, musicians, dancers, etc. Even if they do mathematics, they still behave like artists. They learn their subjects as part of the nature, rather than from teaching staff who are decentralized based on their teaching subjects. These are the ones who enjoy the journey more than the destination. I believe every human enjoys this but gets to take a different approach in serious parts of life.

What happens in a violin or a rhyming poem has something to do with mathematics. They get to see that connection. What’s a language used for has something to do with their biology and geography. They get see that. What happens in the whole cosmos has a deep “rooted” connection with all that happens in our minds and bodies. Some of them arrive here one day.

For these, life happens like one full ride. A seamless stream of happily flowing water.

I’m saying this not because I lived both these lives. I obviously lived one of them and it’s easy to see which one it is. Perhaps that’s what makes people like me qualified to talk about the other. When I speak on these lines, people usually bracket you into a bunch of idealists. They know there are these ideal people living around them. But they hate to accept them as normal and natural.

I’m saying all this with the confidence that, whether you subject a kid to a result-oriented, schooled, controlled and phase-wise life or to the process-oriented, untamed and unconditional life experience, both seem equally experimental, because life throws challenges at both with equal intensity. End of the day, it’s an individual, not owned by anyone, even parents. But one of these two experiment types seems worthier than the other. Which one is it depends on which side one takes.

One of the film directors was on a TV shore recently. He was asked — how he became the star director with his very first film almost 30 years ago and how he got to become a proficient film director without having to use the usual ladder of hierarchy or any apprenticeship or formal training or godfathers and career-makers. He said his first film [and almost every film he made ever since] became a trendsetter because even he wasn’t aware he was creating something like that.

Since I wasn’t schooled about film making, I just did what I genuinely felt like doing and it turned to be different from what anybody did in the past.

The intensity of our involvement into a task is inversely proportional to the amount of formal education we receive. This is something people cannot argue on. Because it is still not a system where metrics/KPIs have been established … luckily.

I Wonder ...

Sometimes, I really feel overwhelmed with too many things to enjoy, to despair at not knowing some of them and to crave for learning some others. Mind goes uncontrollable, there’s some joy and agony together, vibrating my energies both vibrantly and violently. I kind of like it. But I fail utterly to organize those energies.

Some attribute these qualities to my zodiac sign — Gemini. Some call it my inferiority complex. Friends wonder seeing me in these moments and then conclude this is of no use anyway. Family just got used to it. Finally, me? I pass the day and the next day is not the same again.

Should I have done something about those overpouring energies, I would have become something. I never knew that art. Listening to people like Sadhguru is no help until I could actually follow them. To create something out of such energies is really something.

Suddenly every now and then, almost everything around me excites me too much. I start the day by wondering almost everything around me.

The sunlight, the shadows made by it in my room.

Birds chirping, and me waking up to those sounds.

Somebody talking about the irony of how stock market becomes smooth by investors who believe in speculation rather than analysis, and thereby take larger risks.

A. R. Rahman’s school of music, which is also called as “Music Conservatory”.

Popping up physics news with wonderful statements, such as,

“But for much of its existence, the planet was as naked as Earth. While Saturn first formed around 4.5 BILLION years ago, studies suggest the rings are only 100- 200 million years old, tops. That’s younger than some dinosaurs.

So when you think about it, we’re pretty lucky we happened to be around to see those magnificent rings. Really lucky, in fact. Because efforts to study those rings have led us to other discoveries.”

One of my colleagues made a salad look so beautiful yesterday, with coriander leaves as topping. Just the visual captured my brain for a couple of seconds.

My daughter dancing to Rajanikanth song. And my wife’s satire and love being displayed together.

The sprouting blades of grass in a never maintained wetland. And their green shiny edges in reaction to morning sunlight. I see them almost everyday when I walk to my car.

Kids shouting in the neighborhood. And their energy compel me to stare at them. Especially those very small kids who don’t need anything amazing and they manage to keep themselves amazed by everything they physically hold.

The knowledge and intellect of some of the people I meet at work. They fear me, threaten me. But by the end of the day, my philosophy makes them too light to handle. Paradox at every step. Irony in every move.

Numbers grab my attention. Not because I know how to play with them. Quite on the contrary, I look at them like I looked at my teachers when in school. I listen to them as if Nietzsche is talking to me and I can’t make head or tail of what he intends to say. With this fear, my attention for them is too much for me to learn to deal with them. I still like to have that fear and respect. I have some intimate friends who I find extremely good with numbers. Those with grip on this subject always throws a different perspective at me. Not that I didn’t have that perspective at all, but they make it concrete by using the language of numbers. Qualitative thoughts turning into quantitative ideas.

There’s no one thing. No one subject. No one part of my world that I don’t wonder at. From a baby to Benjamin Franklin. From a grass blade to how an airplanes work. From a tiny me to the vast cosmos.

I Wonder.

I wonder at almost everything and everyone.

I don’t want to ask the question if everyone is like this. I want to cherish the fact that I can do this.

Life is beautiful….

And it’s all in the mind!! :-)

Floobits & Flootty for Pair Programming

On othe offset, I don’t really know what Pair Programming means!

At least, I am not confident to say I know what it is. But if I just google search with a question like,

What is pair programming?

It gives me a definition, not so complicated, as shown in the screenshot below.

Not so complicated, except for the part where it says “Agile Software Development Technique”. I don’t want to get into that too. That’s an ocean.

Keep it Simple, Stupid!

Somebody just cried in my ear. So, let me try and say what I understand. Two developers can share their workspace, say a terminal where they run their code, or an editor where they write their code, and work collaboratively. As someone in one of my search results put it, Floobits is like Google Docs for developers. While I write the code or run a command in my terminal, you can not only see it in real time, you can actually change my code or command. Does that make enough sense.

I was really excited when I learned about it and quickly made a couple experiments with friends, and they shared my excitement too.

I thought I should keep it here what I have done so far with Floobits.

In order to be able to use Floobits, and you are the one who owns the workspace and would like to share it with your partner, you need to do the following things.

• Create a Floobits (obviously, free to begin with ;-) account on their website by signing up.
• Once you login with your new credentials, create a public workspace (5 public workspaces and 1 private workspace come with the free account). Make a note of the URL of your new workspace. Something like https://floobits.com/username/workspace-name.
• Go to Settings page and get the following.
  • The contents for the ~/.floorc.json file which you will have to create and save, particularly in your home folder.
• Come back to your terminal and ensure you got python3 with pip3.¹
• Now you need Flootty which is a collaborative terminal, meaning once you have it installed and working, your partner can see and use your terminal.
  • Run pip install Flootty
• That’s pretty much it. Now, you just need to run,

flootty --create --url=https://floobits.com/username/workspace-name

Let us say, you are the partner and would like to share the workspace with its owner. This is what you need to do.

• Get a Floobits account, similar to the owner step above. But you don’t need to create a workspace.
• Get the URL for the owner’s workspace.
• Follow the above steps for
  • creating and saving ~/.floorc.json file in your home folder
  • installing flootty with python3 and pip3
• Finally, run this command to join the owner’s workspace.

flootty --url=https://floobits.com/username/workspace-name

From this point onward, the owner can go back to the terminal and the partner can see it and also use it.

Now there are two ways you can share the terminal and editor.

1. Use web-based workspace provided by Floobits. This is pretty easy. Just go to the URL of the workspace shared with you and click on the terminal on the left menu.
2. Use Atom IDE and install the Atom Floobits Plugin from either inside Atom GUI or from terminal using apm (Atom Package Manager). All that you can do on the browser-based workspace, you can now do it inside Atom. I am not covering this part for this post. Moreover, in both Atom and the browser-based workspace, you can also do video chat. See the gif below from their github repo.

This is very cool and I am sure students should be introduced to tools like this from the very beginning of their learning career.

1: A Quick Note about the way I use Python: I really like Anaconda Python Distribution or at least the Mini Conda package. And I don’t want to make python3 as default on my machine. So … I create an environment with python3 by running conda create --name <some-env-name> python=3. Once you run source activate <some-env-name>, and you will find that for as long as this source is active, your default python is py3 and default pip is pip3.