add 3 of them

1-1-intro.md

@@ -0,0 +1,243 @@
+# Introduction to cloud programming
+
+## Client server architecture
+
+- Definition: Clients make request to server for a resource, and the server
+  running server software make the resource available, though a response.
+    - It is based on **Request/Response** mechanism.
+- Client: Request resources
+- Server: Provide resource
+
+## Distributed System
+
+- Definition: Consists of hardware, software, and data components located at
+  networked computers, which communicate and coordinate their actions using
+  **protocols** and **passing messages**.
+- Distributed Computing: specialized field in cs, dedicated to the study of
+  distributed systems.
+- **Advantages**:
+    - **Modularity**
+    - **Speed** up computation and data processing
+    - Ease of tracking **failure**
+
+### Characteristics
+
+- Concurrency: multiple components working on different parts of the same task
+  at the same time
+- No global clock for time system, use ntp to solve the problem.
+- Autonomous computational entities: failure is **independent**, and is loosely
+  coupled
+- Communicate by passing messages
+
+### Parallel computing vs. Distributed computing
+
+- Parallel computing: many executions are carried out at the same time.
+    - Done by dividing large problem into smaller parts that are carried out by
+      different components at the same time
+    - Use **shared** memory, and exchanges information between processors.
+    - To speed up computation or execution
+- Differences:
+    - Parallel computing:
+        - Sharing: system whose processors use **shared memory** to run
+          processes
+        - Strongly coupled and has resource contention
+    - Distributed computing:
+        - Sharing:have their own individual memory but communicate by passing
+          **messages** among the nodes and processors.
+        - Loosely coupled, and distributed
+
+### Examples
+
+- Internet
+- Intranets
+- Grid computing
+- Peer to peer (p2p) computing
+- Cloud computing
+
+### Architectures
+
+#### Client-server
+
+- 2-tier architecture
+    - Server and clients: one server and many clients
+    - Server has storage, database and run the application
+- 3-tier architecture
+    - The three tiers:
+        - Presentation layer: User interface, runs **web server**
+        - Business logic or application layer: functions are implemented, runs
+          **dynamic content processing and generation** software
+        - Data layer: data management, runs **database management system**(DBMS)
+- Thin and flat client
+    - Thin: client is only the **presentation**, all processing and data storage
+      are provided by server
+        - Heavy lifting is done on server and network
+    - Fat: processing is delegated to the client, foe example **application
+      processing**
+        - Process may be done **both** at client and server
+        - **Balanced** distribution of processing
+
+### Peer to peer systems
+
+- Definition: consists of nodes of computers that perform **both** client and
+  server functionalities and capabilities
+    - **Client** oriented: Peer machines are client computers that is connected
+      to internet
+    - Peer machines **autonomously** join or leave network, no master-slave
+      relationship: **no** **central** coordination, and **no** **global** view
+
+### Flynn's taxonomy
+
+- Definition: 4 classifications for computer architectures
+- SISD: Single Instruction Single Data Stream
+    - Serial computer, no parallelism
+- SIMD: Single Instruction Multiple Data Streams
+    - Some parallelism
+- MISD: Multiple Instruction Single Data Stream
+    - Not popular in practice
+- MIMD: Multiple Instruction Multiple Data Streams
+    - True palallel computation, most widely used
+    - Multiple independent processors executing different instructions on
+      different data streams
+    - Majority of distributed system use MIMD
+    - Basis of supercomputer architectures
+
+## Cloud Computing
+
+### History of Cloud computing
+
+- Started as a concept of "Utility Computing", which you can use as public
+  utility.
+- Grid Computing:
+    - Started the concept, in 1995 to mimic the electricity grid that provides
+      electric as a utility on demand basis
+    - Large scale distributed systems:
+        - Collection of resources from distributed locations
+    - General purpose middleware
+        - Interacts with heterogeneous systems
+- The above gave birth to cloud computing
+
+
+### Definition
+
+- A computing infrastructure, that consists of shared pool of **virtualized**
+  hardware, software and data that are **provided** as a **services**, which
+  scales on the demand
+- A form of **distributed** computing, that enables computational resources to
+  be provided as a service
+- Infrastructure as software
+
+### Features:
+
+- Enabled by **internet**
+- Use middle-ware to connect systems
+- Utility based:
+    - Don't own or maintain the resources, but subscribe
+- Usage on **demand**
+- Elastic, can scale up and down
+- **Service Oriented Architecture**
+- Cheaper and save time
+
+### Architecture
+
+- Front end
+- Cloud based delivery
+- Back end
+
+## Cloud Deployment Models
+
+###  Public Cloud
+
+- Advantages:
+    - Flexible
+    - Reliable
+    - Highly **scalable**
+    - Low cost
+    - Place independence
+- Disadvantages:
+    - Less secures
+    - Not **customizable**
+
+### Private cloud
+- Advantages:
+    - Highly **private** and **secured**
+    - More **control** 
+- Disadvantages:
+    - Poor scalability
+        - Scaled within hosted resources
+    - Costly: 
+        - secured
+        - More features
+    - Inflexible pricing
+    - Restriction, hard to expose globally
+
+## Cloud Service Models
+
+- Common:
+    - SaaS: Software as a service: least control
+    - PaaS: Platform as a service
+    - IaaS: Infrastructure as a service: most control
+- Incommon:
+    - DBaaS: database
+    - Naas: network
+    - SECaaS: Security
+    - DaaS: Desktop
+
+## Technologies for cloud computing
+
+- Internet
+    - web service
+- Virtualization technologies
+- Data center
+- Networking
+- Security
+- Physical hardware
+
+## Services:
+
+- Amazon web services: AWS
+- Microsoft Azure
+- Google Cloud
+- Alibaba Cloud
+- Tencent Cloud
+- Baidu Cloud
+- IBM Cloud
+- Oracle Cloud
+- https://salesforce.com
+
+## Applications and Job opportunities
+
+### Jobs
+
+- Big Data and Data analytics
+- E-Commerce
+- Computationally and data intensive applications
+- AI
+- IoT
+- Software Development
+
+### AWS Certification Exam
+
+- Professional: 2 years
+- Associate: 1 year
+- Foundational 6 month
+- Speciality: Technical experience
+
+### MS Exam
+
+- Expert
+- Associate
+- Fundamental
+- Speciality
+
+### Google Cloud Certification Exams
+
+- Data Engineer
+- CLoud Architect
+- Cloud Developer
+- CLoud devops Engineer
+- Cloud Security Engineer
+- Collaboration engineer
+- Cloud network engineer
+- Cloud engineer
+- G suite
+- Hybrid multi-cloud

1-2-sla-virtualization.md

@@ -0,0 +1,307 @@
+# SLA and Virtualization
+
+## SLA
+
+### Definition
+
+- Service Level Agreement: a service-based legal agreement between a
+  **provider** and a **subscriber**
+- Accepted before a cloud service can be used
+- Content
+    - Provider's responsibility
+    - Subscriber's responsibility, and entitlements
+    - Billing and accounting system
+
+### Types
+
+- Service based SLA:
+    - Offer shared resources, used by many customers
+    - Offered to cloud users
+- Customer based SLA
+    - Agreement between individual customer or group, to provide non-shared
+      **specific** services
+- Multi-level SLA:
+    - Corporate level: generic and cover all user of an organization
+    - Customer level: specific group of user
+    - Service: specific type of service
+
+### Boundaries
+
+- Organizational boundary
+    - Physical perimeter, owned and governed by organization or cloud provider
+- Trust boundary
+    - When cloud user extend trust beyond its physical boundary, and include
+      resources from the cloud
+
+### Characteristics of cloud
+
+- On-demand usage: use more cloud resources only when they need it
+- Multi-tenancy aka. Resource pooling: Resources are **instances** that can be
+  used by many users
+- Scalability: Able to automatically scale up or down based on the settings and
+  agreements
+- Ubiquitous access: Can be accessed everywhere with lots of devices
+- Measured usage: keep track and record the amount of IT resources used by
+  consumers
+    - Used for accounting and auditing
+- Resilience: distribute and replicate resources to provide redundancy, and fail
+  over mechanism
+
+### QoS
+
+- SLA is based on **Quality of Service**
+- Combination of metrics relating to requirements, and satisfaction of services
+  offered by a provider.
+- Quantitative or qualitative
+- Parameters:
+    - **Reliability** availability
+    - **Security** and trust
+    - **Scalability**
+    - **Data** integrity and data protection
+    - **Resilience**
+    - Data **recovery** time
+    - Support
+
+## Virtualization
+
+### Definition
+
+- The process of creating the **virtual** version of hardware, database,
+  operating system and application from real copies, using virtual machine
+  **monitor** to emulate the physical functionalities.
+- Multiple virtual machines are **multiplexed** in to the same hardware, to
+  serve as an abstraction of physical OS
+- Guest OS is not bound to the physical hardware.
+- **Guest** OS runs on **host** OS, and they can be different
+
+### Levels of virtualization
+
+- Instruction Set Architecture level: ISA **emulation**, which is used to run
+  many legacy codes written for a different hardware configuration, like
+  _Dynamo_
+- Hardware level: VMs are located directly on top of **hardware**, allows the
+  virtualization of each of the hardware components, which could be the
+  input-output device, the memory, the processor, etc, like _VMWare_
+- Operating system level: an operating system (OS) virtualization paradigm in
+  which the kernel allows the existence of multiple isolated user space
+  instances, like _Jails_
+- Library level: translate API calls to user libraries, like _vCUDA_
+- Application level: allows high-level language programs to be compiled easily.
+  java's _jvm_
+
+### Types of Virtualizations
+
+- Desktop: Create desktop computer from physical desktop machine
+- Hardware: Virtual computer with OS that represents and act as a real physical
+  computer.
+- Nested virtualization: Virtual machine inside another one
+- CPU virtualization: Virtual CPU are created to reduce stress on host
+- Memory virtualization: virtual memory is created and run on the VM, controlled
+  by the virtual CPU of VM, but can't access memory address outside virtual
+- IO virtualization: Route IO request between virtual devices and physical
+  hardware
+
+### Virtual clusters
+
+- Physical cluster: interconnected by physical network
+    - Failure may cause a falling physical node
+- Virtual cluster: VMs intereconnected by a virtual network across several
+  physical networks
+    - Node can be either physical or virtual
+    - Cluster VMs can run different OS
+    - To enhance **utilization** and **flexibility**
+    - Can be freely replicated, and is **fault** tolerant easy to recover
+    - Nodes can grow or shrink
+    - Failure won't cause shutdown to host machine
+
+### Migration of VM
+
+#### Introduction
+
+- VM can be moved from one cluster to another, which **isn't possible** with
+  physical hosts.
+- Four ways:
+    - Guest-based manager, where cluster manager is in guest vm
+    - Host based manager: supervises the guest system, and can restart guest
+      system remotely
+    - Independent cluster manager on both host and guest
+    - Use integrated cluster manager
+
+#### Six (five) steps of VM migration
+
+- Start migration (for some reason this is counted as step 0 and 1???)
+    - Determine VM to migrate
+    - Determine destination host
+- Transfer memory
+    - State is stored in memory, so transfer memory first
+- Suspend VM and copy remaining data
+    - Execution is suspended when last memory is transfered.
+    - Non memory components like network and cpu are sent to destination
+- Commit
+    - After all data, VM reloads and continue execution
+- Activate
+    - Network is redirected to new VM
+    - remove the original VM from source host
+
+### Mechanism for cloud infra: building blocks
+
+- Environment:
+    - Logical network perimeter: Virtual network boundary, encompass and isolate
+      a group of resources, that may be **physically** distributed., to _isolate
+      cloud resource from unauthorized users_
+    - Virtual Server: behaves exactly like a physical one, and running on the
+      same physical server which is shared by multiple cloud users
+    - Resource replication: create multiple of same resource to various
+      locations, physically distributed around the world.
+    - Ready made environment: provided by PaaS (platform as a service)
+    - Failover system: use clustering to provide redundancy
+    - Hypervisor: generate virtual server instances
+    - Multi device broker: converts and exchanges information
+    - Resource cluster: cloud resources are geographically distributed and group
+      resources and instances together
+    - State management database, used to temporarily maintain state data for
+      software programs to improve performance.
+- Scaling
+    - Automated Scaling listener: monitors communication between consumers and
+      services for dynamic scaling up or down.
+    - Load balancer: cloud runtime mechanism that distributes workload: data
+      processing, computation, network traffic across two or more cloud
+      resources, for better performance( automated horizontal scaling)
+- Monitoring
+    - Cloud usage monitor: Autonomous software used for collecting and
+      processing cloud resource usage data.
+    - SLA monitor: monitors the performance to ensure they are fulfilling the
+      Quality of Service requirement
+    - Pay per use monitor: monitor the usage and calculate the price according
+      to the agreement
+    - Audit monitor: track data for network, to confirm to regulatory obligation
+
+### Different hypervisors
+
+#### Microsoft virtualization technology
+
+- parent partition: logical hard drive for running hypervisor layer, and nesting
+  child partitions
+- Child partitions: Logical hard drive responsible for running isolated OSes
+- Virtualization stack Collections of resources including the hypervisor, which
+  makes up the Microsoft hyper V
+- Virtualization service provider: parent partition side, server that handles IO
+- Virtualization child provider: child partition component, that sends request
+  to VSP
+- VMBus: send requests and data between VMs
+- WMI: Windows management instrumentation, manages virtual devices
+- Windows kernel: manages IO request from parent and child
+- IHV: hardware drivers
+- VM worker process: run and configure one VM each
+- Hypervisor
+    - Manages a minimum set of **hardware** components
+    - **Isolate** component, to enhance **loose coupling**
+    - Provide **security**
+    - Enhance **performance**
+- [Image](./assets/1-2-microsoft-virtualization-technology.png)
+
+#### Xen Virtualization Technology
+
+- [Image](./assets/1-2-xen.png)
+- Host Hardware: real hardware
+- Xen Hypervisor: lean software layer, that runs on **host hardware** and
+  manages the **CPUs, memory, IO devices**
+- Guest Domains / Virtual Machines
+- Control Domain, or domain 0: special virtual machine with privilege to access,
+  and handles IO functions and VM's interactions
+- Toolstack: control stack on domain **0**, enables user to manage **virtual**
+  machines
+
+### Virtualization Technologies
+
+- host components: real hardware
+- guest components: virtual system created from host machine, includes os,
+  network, databases and servers
+- Hypervisor, or virtual machine manager: manages virtual machines
+- Networking technologies
+- Web services: manager over the internet
+- Front end systems: interface between user and virtual machine
+
+### Advantages and challenges
+
+#### Advantages:
+
+- Deployment:
+    - Less **cost**, reduction in cost of hardware, energy, cooling , expertise
+      and administration
+    - Less time needed
+    - Creating servers on different system is **easy**
+- Maintainence:
+    - **Scalability**
+    - **Efficiency**
+    - **Reliability**
+    - Fast data disaster **recovery**
+    - Backups and migrations are **easy**
+
+#### Challenges
+
+- **Security** of virtual machines
+- Need **expertise** to manage virtual machines
+- **Reliant** back up systems are needed
+- Failures of **host** will affect the guest system.
+- **Performance** may not be as good as using physical ones
+
+### Future of Virtualization
+
+- Is and will be the **dominant** component of cloud computing
+- Decoupling of application will **increase**
+- **Less** energy consumption by optimizing OS and VMM
+- **Dominant** in IT market
+- Even more **scalable**
+- Better **security**
+
+### Major classes of hardware virtualization
+
+#### Full virtualization, Hardware virtual machine
+
+- Complete simulation of actual hardware, so that OS and software doesn't need
+  any modifications.
+- AWS Support
+    - HVM AMI (Amazon Machine Images), in a fully virtualized environment
+    - Can run OS without modification
+    - HW extension is support
+    - Instance type support: all current
+    - Regioin: all
+
+#### Paravirtualization
+
+- Allows guest applications to be executed, as if they are running on a separate
+  system, but hard ware environment is not simulated
+- Guest need to be modified
+- AWS Support
+    - PV AMI: special bootloader is used
+    - Can run on host hardware without explicit support for virtualization
+    - Hardware extention is not supported
+    - Some instance types are supported
+    - Some regions are supported
+
+### Layering and Virtualization
+
+#### Layering
+
+- Definition: approach to manage system complexity
+- Goals:
+    - Simplify description of subsystem, to **abstract through interface**
+    - Minimize interaction between subsystems
+    - Enable us to independently manage each subsystem
+- Layering in computers:
+    - Hardware
+    - Software
+        - OS
+        - Library
+        - Applications
+
+#### Interfaces in virtualization
+
+- ISA (Instruction Set Architecture): Instruction Set Architecture: boundary
+  between hardware and software
+- ABI (Application Binary Interface): Allow application and library modules to
+  access hardware, invokes **system calls**
+- API (Application Program Interface): Is a set of rules or protocols that
+  enables software applications to communicate with **each other** to exchange
+  data, features and functionality

1-3-economy.md

@@ -0,0 +1,132 @@
+# The economics of cloud computing
+
+## Cloud Computing Service Model (Not important)
+
+### Service oriented Technology
+
+- Properties
+    - Logical view: abstraction of actual processes
+    - Message orientation: uses message to communicate
+    - Description orientation: described by machine executable metadata
+    - Granularity: use small operation, in order to perform complex tasks
+    - Network orientation: offered through networks
+    - Platform neutrality: standardized format
+- Examples: SaaS, PaaS, IaaS
+
+### Objectives:
+
+- Shift computing, thus management and maintainence
+- High QoS
+- Transparent, and generic standards to boost acceptability
+
+## Economic models
+
+### Traditional cost model
+
+- Upfront capital costs (CAPEX, capital expenditure): money spent at the beginning of business to get the
+  equipment
+- Variable cost (OPEX, Operational expenditure): operational cost
+- Calculation:
+    - Upfront capital costs + Variable costs
+
+### Cloud cost model
+
+- Calculation
+    - Variable costs (no upfront cost)
+    - Traditional IT cost minus the upfront capital costs (This sounds stupid
+      and absurd, I hope the profs are not going to make a quiz on this)
+- Optimizing
+    - AWS Trusted Advisor: online service, that helps suggesting ways to
+      optimize costs
+    - Calculators:
+        - TCO: Total cost of ownership calculator, estimate the cost savings
+          when using AWS, compared to traditional IT systems
+        - Simple Monthly Calculator: estimate your AWS monthly bills, based on
+          business requirements
+
+### Examples
+
+- Amazon cloud options:
+    - Reserved:
+        - Reserved capacity: reserve for 1 to 3 years to get discount
+        - Reserved instances:
+            - All-Upfront RI: Pay everything at once to get huge discount
+            - Partial-Upfront RI: Pay part of the amount and get less savings
+            - No-Upfront RI: Pay nothing upfront, little saving
+    - On-demand instance: Pay as you go
+    - Dedicated instance: run on physical server
+    - Free: 12 months of free EC2
+
+### Motivation for both parties
+
+#### Subscribers:
+
+- Cost savings
+    - Power
+    - Support, Maintainence
+    - Software licenses
+    - Increased profit
+- Fancy words
+    - Innovation
+    - Productivity
+    - Competitive advantage
+    - Elastic
+- Development experience
+    - Automation
+    - Reliability
+    - Scalability
+
+#### Providers
+
+- Shared resource, no waste
+- Location independent
+- Variety of customers that can serve
+
+## Competition
+
+- Offerings
+    - Free and educational
+    - Reserved
+    - Novelties: glacier file storage
+- Assurance
+    - Risk mitigation
+    - Reputation
+    - Size
+
+## All kinds of models
+
+### Service delivery
+
+- Definition: pre-packaged combination of IT resources, offered by cloud
+  provider as service
+- Examples: [previous slide](/1-1-intro.md#cloud-service-models)
+
+### Deployment
+
+- Private cloud: on premises cloud
+- Public cloud
+- Community cloud: infra is shared between several organizations
+- Hybrid cloud: public cloud with a private cloud, or on-premises data center
+
+## Challenges and opportunities:
+
+### Challenges
+
+- Demand on features and support
+- [Competition](#competition)
+- [QoS](/1-2-sla-virtualization.md#qos)
+- [SLA](/1-2-sla-virtualization.md#sla)
+
+### Opportunities
+
+- Jobs:
+    - AWS Cloud Practitioner
+    - AWS Solution Architect
+    - AWS Developer
+    - AWS DevOps Engineer
+    - AWS SysOps Administrator
+    - AWS Security Expert
+    - AWS Big Data Expert
+    - AWS Networking Expert
+    - AWS Data Engineer
+    - AWS Data Scientist