CS385 – Computer Architecture

Spring-2009

Catalog description: The architecture of the computer is explored by studying its various levels: physical level, operating-system level, conventional machine level and higher levels. An introduction to microprogramming and computer networking is provided.

Course Prerequisites: CS 354

Prerequisites by topic:

• Basic skills in software design and programming
• Assembly language programming and basics of computer organization
• Digital systems design
• Boolean algebra and discrete mathematics

1. MIPS instruction set
2. Computer arithmetic and ALU design
3. Datapath and control
4. Pipelining
5. Memory hierarchy, caches and virtual memory
6. Interfacing CPU and peripherals, buses
7. Multiprocessors, networks of multiprocessors, parallel programming
8. Performance issues

• Understand the fundamentals of different instruction set architectures and their relationship to the CPU design.
• Understand the principles and the implementation of computer arithmetic.
• Understand the operation of modern CPUs including pipelining, memory systems and busses.
• Understand the principles of operation of multiprocessor systems.
• Design a CPU (multicycle implementation) by a given specification using HDL.
• Write simple parallel programs.

Required software:

1. Icarus Verilog: HDL compiler and simulator, available on the Patterson and Hennessy's book CD or at http://www.icarus.com/eda/verilog/. Note about installation: don't use folder names that include spaces (like Program Files). Read book sections B.4 and 5.8 for using HDL.
2. Other simulators that may be used for experiments (note that the project should be done with a Verilog simulator):
• Digital Works 2.0 (freeware)
• Verilog HDL (Book CD of Morris Mano, Digital Design, Third Edition, Prentice Hall, 2002, ISBN 0-13-062121-8 )
3. SPIM simulator: A free software simulator for running MIPS R2000 assembly language programs available for Unix, DOS, and Windows.

Class Participation: Active participation in class is expected of all students. Regular attendance is also expected. If you must miss a class, try to inform the instructor of this in advance.

Honesty policy: It is expected that all students will conduct themselves in an honest manner (see the CCSU Student handbook), and NEVER claim work which is not their own. Violating this policy will result in a substantial grade penalty, and may lead to expulsion from the University.

Grading: Grading will be based on one programming assignment (5%), a midterm test (25%), a final exam (25%) and a semester project (45%, including progress reports and the final documentation). The letter grades will be calculated according to the following table:
 

A	A-	B+	B	B-	C+	C	C-	D+	D	D-	F
95-100	90-94	87-89	84-86	80-83	77-79	74-76	70-73	67-69	64-66	60-63	0-59

Unexcused late submission policy: Submissions made more than two days after the due date will be graded one letter grade down. Submissions made more than a week late will receive two letter grades down. No submissions will be accepted more than two weeks after the due date.

Tentative schedule of classes and assignments

1. January 27:  Introduction: Computer Architecture = Instruction Set Architecture + Machine Organization
2. February 3:  MIPS Instructions: arithmetic, registers, memory, fecth&execute cycle
3. February 5:  MIPS Instructions: control and addressing modes
4. February 10:  Computer arithmetic and ALU design: representing numbers, arithmetic and logic operations
5. February 12: Assignment 1 due (5 pts.).
6. February 17: ALU design: full adder, slt operation, HDL design, carry lookahead
7. February 19: ALU design: multiplication, representing floating point numbers
8. February 24: The Processor: Building a datapath
9. February 26: The Processor: Control (single cycle approach)
10. March 3: Progress Report #1 due (10 pts.): ALU, Register File. Use template files: ALU4.vl (extend it to 16 bit) and regfile.vl (change the D flip-flops with 16-bit registers and redesign mux4x1 using gate-level modeling and extend it to 16-bit data).
11. March 5: Multicycle approach to processor control
12. March 10: Implementing finite state machine control, Microprogramming
13. March 12: Using a Hardware Description Language to Design and Simulate the MIPS processor
14. March 17: Turing machines
15. March 31: Introduction to pipelining
16. March 31: Progress Report #2 due (10 pts.) - see Semester Project for details.
17. April 2: Solving pipeline hazards
18. April 7: Implementing pipeline datapath and control
19. April 9: Midterm Test (25 pts.) due
20. April 9: Implementing data and branch hazards control
21. April 14: Review of Datapath, Control and Pipelining
22. April 16: Memory hierarchy, The Basics of caches
23. April 21: Improving cache performance
24. April 28: Progress Report #3 due (10 pts.) - Branching logic, Execution control. See the project description for more details.
25. May 5: Virtual Memory basics, Virtual Memory optimization, A general framework of memory hierarchies
26. May 7: Interfacing Processors and Peripherals - Buses, Interfacing I/O devices to Memory, CPU and OS
27. May 12: Multiprocessors, Networks of muiltiprocessors
28. The role of performance
29. May 21: Final Exam due (25 pts.)
30. May 21: Semester Project due (15 pts.)

CS385 – Computer Architecture, Lecture 1

Lecture Notes

1. Levels of Abstraction
2. Computer Architecture = Instruction Set Architecture + Machine Organization
3. Instruction Set – The Software Hardware Interface
4. Levels of Computer Architecture in More Depth
• Software:
• Application
• Operating System
• Firmware
• Instruction Set Architecture:
• Organization of Programmable Storage
• Data type and Structures: encodings and machine representation
• Instruction set
• Instruction Formats
• Addressing Modes and Accessing Data and Instructions
• Exception Handling
• Hardware:
• Instruction Set Processing
• I/O System
• Digital Design
• Circuit Design
• Layout
5. Basic Components of a Computer
• Processor: Datapath and Control
• Memory
• I/O
6. Computer Organization
• Capabilities and Performance of the Basic Functional Units
• The Way These Units are Interconnected
• Information Flow between components
• Information Flow Control

CS385 – Computer Architecture, Lecture 2

Lecture Notes

1. Design goal: maximize performance and minimize cost. Primitive (low level) and very restrictive instructions (fixed number and type of operands).
2. Design principles:
• Simplicity favors regularity (uniform instruction format)
• Smaller is faster (only 32 registers)
• Good design demands a compromise (I-type instructions)
3. MIPS arithmetic: 3 operands, fixed order, registers only.
4. Using only registers: R-type instructions.
5. Registers: 32-bits long, conventions.
6. Memory organization: words and byte addressing.
7. Data transfer (load and store) instructions. Example: accessing array elements.
8. Translating C code into MIPS instructions – the swap example.
9. Machine Language: instruction format, I-type (Immediate) format for data transfer
10. Stored program concept: programs in memory, fetch&execute cycle
• Von Neumann Architecture
• CPU, Memory System, I/O system
• Stored program concept: programs in memory, fetch&execute cycle
• Instructions are executed sequentially
• Turing machines
• Turing machines
• Alan Turing web page
• A Turing Machine Applet
• Non-Von Neumann Architecture:
• Various parallel and multiprocessor architectures (see book chapter 9)
• In broader sense: NN, GA etc.
• The Modern History of Computing
• John von Neumann

CS385 – Computer Architecture, Lecture 3

Lecture Notes

1. Implementing the C code for if in MIPS: conditional branch.
2. Implementing the C code for if–else in MIPS: unconditional branch
3. Simple for loop
4. Check for less-than: building a pseudoinstuction for branch if less-than.
5. Addressing in branch instructions: PC-relative and pseudodirect.
6. Constants: use of immediate addressing (constants as operands – addi, slti, andi, ori)).
7. 32-bit constants – manipulate upper 2 bytes separately (load upper immediate)
8. Summary of MIPS addressing: register (add), immediate (addi), base or displacement (lw), PC-relative (bne), pseudodirect (j).
9. Alternative approaches: IA-32

CS385 – Computer Architecture, Lecture 4

Lecture Notes

1. Representing numbers: sign bit, one's complement, two's complement.
2. Arithmetic: addition, subtraction, detecting overflow.
3. Logical operations: shift, and, or.
4. Basic ALU building components: and-gate, or-gate, inverter, multiplexor.
5. ALU for logical operations.
6. ALU for add, and, or.
7. Supporting subtraction

Tutorials and practice quizzes on two’s complement numbers:

• http://chortle.ccsu.edu/AssemblyTutorial/Chapter-08/ass08_1.html (tutorial)
• http://chortle.ccsu.edu/AssemblyTutorial/Chapter-08/ass08quiz.html (quiz)

CS385 – Computer Architecture, Lecture 5

1. Implementation of a full adder:
• Carry out logic
• Result logic: using 'and', 'or' and inverter and using xor-gate.
2. Supporting set on less-than (slt).
3. Test for equality (needed for branching)
4. Designing the ALU in Verilog
5. Carry Lookahead

CS385 – Computer Architecture, Lecture 6

Lecture Notes

1. Implementing multiplication:
• Using 64-bit adder;
• Using 32-bit adder for the upper 32-bit of the product;
• Avoiding the use of the multiplier register.
2. Floating point numbers
• Scientific notation: (-1)^sign * significand * 2^exponent
• Range and precision (overflow and underflow).
• IEEE 754 floating point standard - allows integer comparison:
• normalized representation
• implicit leading 1
• exponent is biased: exponent in [0..0 (most negative), 1..1 (most positive)]
• bits of the significand represent the fraction between 0 and 1.
• (-1)^S * (1 + s1*2^-1 + s2*2^-2 + ...) * 2^(exponent-bias)
• Problems with floating point arithmetic

• http://chortle.ccsu.edu/AssemblyTutorial/Chapter-30/ass30_1.html (tutorial)
• http://chortle.ccsu.edu/AssemblyTutorial/Chapter-30/ass30quiz.html (quiz)

CS385 – Computer Architecture, Lecture 7

Lecture Notes

1. Abstract level implementation:
• Instruction memory
• Program counter
• Register file
• ALU
• Data memory
2. Basic building elements
• Combinational logic
• State elements: D-lathes and D flip-flops
• Clocking methodology: edge triggered
3. Fetching instructions and incrementing the program counter
4. Register file and execution of R-type instructions
5. Datapath for lw and sw instructions (add data memory and sign extend)
6. Datapath for branch instructions

CS385 – Computer Architecture, Lecture 8

Lecture Notes

1. ALU control: mapping the opcode and function bits to the ALU control inputs
2. Designing the main control unit
3. Operation of the Datapath (single-cycle implementation):
• R-type instructions
• Load (store) word
• Branching instructions
4. Problems of the single-cycle implementation

CS385 – Computer Architecture, Lecture 9

Lecture Notes

1. Basic principles:
• Breaking up instruction into steps
• Reusing functional units in different steps
• Storing intermediate results
• Need for controlling the sequence of steps
2. Execution steps:
• Instruction fetch
• Instruction decoding and register fetch
• Instruction dependent (memory reference, R-type execution or branch)
• R-type completion or memory access
• Memory read completion
3. Implementing the control unit - finite state machine

• How many cycles will it take to execute this code?
lw $t2, 0($t3)
lw $t3, 4($t3)
beq $t2, $t3, Lbl      #assume not
add $t5, $t2, $t3
sw $t5, 8($t3)
Lbl:
...
• What is going on during the 8th cycle of execution?
• In what cycle does the actual addition of $t2 and $t3 takes place?

CS385 – Computer Architecture, Lecture 10

Lecture Notes

1. ROM implementation
• Single ROM: 10-bit address, 20-bit word, 2^10*20=20K
• Two ROM's: 2^10*4 (Op+State-> Next state) + 2^4*16 (State -> Control)
2. Programmable Logic Array (PLA)
• Representing truth table as a sum of products
• AND-gates and OR-gates arrays
3. Implementing the Next-step function with a Sequencer
4. Implementing datapath control by microprogramming
• Microinstruction specification at symbolic level
• Microinstruction format
• Executing microinstructions
5. Microprogramming vs. ROM and PLA implementation
6. Handling exceptions and interrupts
• Types of exceptions and interrupts
• Handling exceptions
• Extending the datapath control to detect exceptions

CS385 – Computer Architecture, Lecture 11

1. Behavior model of MIPS (mips.vl)
2. Project version (two stage control) of MIPS (mips2.vl). Changes needed to complete the project:
• Adjust the word size (ALU, registers, memory etc.) to 16 bit
• Replace the behavior model of fetching with structural design
• Implement the execute phase using structural design
• Write a complete program for testing and test the CPU

• Implement the addi instruction by:
• adding a new row to the single cycle main control table
• adding a new column to the multicycle control table
• modifying the microprogram
• modifying the HDL code (mips.vl)

Turing machines

1. Turing machines
2. Alan Turing web page
3. A Turing Machine Applet

CS385 – Computer Architecture, Lecture 13

Lecture Notes

1. Pipelining by analogy (laundry example):
• Pipelining helps throughput of the entire workload
• Multiple tasks operating simultaneously and using different resources
• Potential speedup = number of pipe stages
• The pipeline rate is limited by the slowest stage
• Unbalanced lengths of pipe stages reduces the speedup
• The time to "fill" the pipeline and the time "drain" it reduces the speedup
• Stall for dependencies
2. Five stages of the load MIPS instruction
3. The pipelined datapath
4. Single cycle, multiple cycle vs. pipeline
5. Advantages of pipelined execution
6. Problems with pipelining (pipeline hazards)
• Structural hazards
• Data hazards
• Control hazards

CS385 – Computer Architecture, Lecture 14

Lecture Notes

1. Structural hazards: single memory
2. Control hazards:
• Stall: wait until decision is clear
• Predict: fixed prediction (e.g. fail), dynamic prediction (based on history)
• Delayed brach (software solution):
add $4, $5, $6            beq $1, $2, $40
beq $1, $2, 40     ==>    add $4, $5, $6
lw $3, 300($0)            lw $3, 300($0)
3. Data hazards (dependecies backwards in time):
• Forwarding (bypassing)
• Reordering code
lw $t0, 0($t1)               lw $t0, 0($t1)
lw $t2, 4($t1)      ==>      lw $t2, 4($t1)
sw $t2, 0($t1)               sw $t0, 4($t1)
sw $t0, 4($t1)               sw $t2, 0($t1)
4. Designing a pipelined processor

CS385 – Computer Architecture, Lecture 15

Lecture Notes

1. Splitting datapath into stages: using registers to store parts of the instruction
2. Transferring data forward and backward between the stages: lw example
3. Corrected datapath: storing rd for the write back stage.
4. Graphically representing pipelines: multiple-clock-cycle vs. single-clock-cycle diagram
5. Pipeline control:
• IF: no control signals to store for later stages (they are always asserted).
• ID: no control signals to store for later stages (they are always asserted).
• EX: set RegDst, ALUOp, ALUSrc
• MEM: set Branch, MemRead, MemWrite
• WB: set MemtoReg, RegWrite
6. Datapath with control
7. Example: running this code through the pipeline in 9 cycles.

lw   $10, 20($1)
sub  $11, $2, $3
and  $12, $4, $5
or   $13, $6, $7,
add  $14, $8, $9

CS385 – Computer Architecture, Lecture 16

Lecture Notes

1. Detecting data dependencies
• EX/MEM.Rd = ID/EX.Rs

EX/MEM.Rd = ID/EX.Rt
• MEM/WB.Rd = ID/EX.Rs

MEM/WB.Rd = ID/EX.Rt
2. Forwarding
• if (EX/MEM.RegWrite and EX/MEM.Rd = ID/EX.Rs)ForwardA = 10

if (EX/MEM.RegWrite and EX/MEM.Rd = ID/EX.Rt)ForwardB = 10
• if (MEM/WB.RegWrite and MEM/WB.Rd = ID/EX.Rs)ForwardA = 01

if (MEM/WB.RegWrite and MEM/WB.Rd = ID/EX.Rt)ForwardB = 01
3. Data hazards and stalls

If (ID/EX.MemRead and
    (ID/EX.Rt = IF/ID.Rs or
     ID/EX.Rt = IF/ID.Rt))
   stall the pipeline
4. Branch hazards
• Reducing the delay of branches - move up the address calculation (move the adder) and the branch decision (add XOR and AND gates)
• Assuming the branch will not be taken
• Flashing instructions in IF, ID, and EX stages, if the branch is taken
5. Advanced pipelining
• Superpipelining
• Superscalar
• Dynamic scheduling

CS385 – Computer Architecture, Lecture 17

Lecture Notes

Datapath

1. Abstract level implementation:
• Instruction memory
• Program counter
• Register file
• ALU
• Data memory
2. Basic building elements
• Combinational logic
• State elements: D-lathes and D flip-flops
• Clocking methodology: edge triggered
3. Basic operations
• Instructions fetch
• Accessing register file and execution of R-type instructions
• Datapath for lw and sw instructions (add data memory and sign extend)
• Datapath for branch instructions

Control

1. ALU control: mapping the opcode and function bits to the ALU control inputs
2. Designing the main control unit
3. Operation of the Datapath (single-cycle implementation):
• R-type instructions
• Load (store) word
• Branching instructions
4. Problems of the single-cycle implementation
5. Multicycle Approach to Processor Control
6. Basic principles of the Multicycle Approach to Processor Control
• Breaking up instruction into steps
• Reusing functional units in different steps
• Storing intermediate results
• Need for controlling the sequence of steps
7. Execution steps:
• Instruction fetch
• Instruction decoding and register fetch
• Instruction dependent (memory reference, R-type execution or branch)
• R-type completion or memory access
• Memory read completion
8. Finite state machine control
9. Microprogramming

Pipelining

1. Basic principles of pipelining
• Pipelining helps throughput of the entire workload
• Multiple tasks operating simultaneously and using different resources
• Potential speedup = number of pipe stages
• The pipeline rate is limited by the slowest stage
• Unbalanced lengths of pipe stages reduces the speedup
• The time to "fill" the pipeline and the time "drain" it reduces the speedup
• Stall for dependencies
2. MIPS pipelining: the five stages of the lw instruction
3. Problems with pipelining:
• Structural hazards
• Data hazards
• Control hazards
4. Designing a pipelined processor
5. Transferring data forward and backward between the stages
6. Pipeline control
7. Implementing data and branch hazard control
• Detecting data dependencies
• Forwarding
• Data hazards and stalls
• Branch hazards
8. Advanced pipelining

CS385 – Computer Architecture, Lecture 18

Lecture Notes

1. Memory technologies and trends
2. Impact on performance
3. The need of hierarchical memory organization
4. The principle of locality
5. Memory hierarchy terminology
6. Basics of RAM implementation
• SRAM:  D-latches, three-state buffers, address decoders, two level addressing
• DRAM:  DRAM cell, refreshing
• Error detection and correction

CS385 – Computer Architecture, Lecture 19

Lecture Notes

1. Direct-mapped cache
2. Accessing a cache
• Cash index
• Tag
• Valid bit
3. Writing to the cache (write-through and write-back schemes)
4. Handling cache misses
• Read miss: load the word from memory
• Write miss: write both to the cache and to the memory (using write buffer)
5. Example: DECStation 3100 cache
6. Spatial locality caches: keeping consistency on write
7. Main memory organization

CS385 – Computer Architecture, Lecture 20

Lecture Notes

1. Measuring cache performance
• Stall clock cycles = Instructions * Miss rate * Miss penalty
• Example 1 (reducing CPI):
• 2% - instruction miss; 4% - data miss; 36% - lw/sw; 40 cyc - miss penalty.
• 2 CPI => CPI_stall = 3.36, i.e. perfect cache is 1.68 times faster.
• 1 CPI => CPI_stall = 2.36, i.e. perfect cache is 2.36 times faster.
• Example 2: doubling clock rate => 80 cyc - miss penalty, CPI_stall = 4.75, Performance: 3.36/(4.75/2) = 1.41 faster with stalls, 2 times faster without stalls.
• Conclusion: cache penalties increase as the machine becomes faster
2. Flexible placement of blocks in the cache
• Direct mapped: Cache index = (Block address) modulo (Cache size); no search; small tag.
• Set associative: Cache index = (Block address) modulo (Number of sets in cache); search the set; larger tag.
• Fully associative: Cache index is not determined; search the whole cache; tag = address.
3. Locating a block in the cache: N-way cache requires N comparators and N-way multiplexor
4. Choosing which block to replace: least recently used
5. Multilevel caches

• Write a sequence of memory references for which:
• the direct mapped cache performs better than the 2-way associative cache;
• the 2-way associative cache performs better than the fully associative cache.
• Exercises 7.9, 7.10 (page 556), 7.29 (page 558).

CS385 – Computer Architecture, Lecture 21

Lecture Notes

1. The need of VM
• Many programs (processes) can use a single memory
• Use a memory exceeding the size of the main memory
2. VM organization and terminology: virtual address, physical address, page, page offset, page fault, memory mapping (translation).
3. Design decisions motivated by the very high cost of page faults:
• Large pages (4K-64K)
• Reducing page fault penalties: fully associative VM
• Software management of page faults
• Write-back instead of write-through
4. Addressing pages:
• Page table, page table register
• Processes (active, inactive) and page tables
• Page faults
• Replacing pages: LRU, reference (use) bit
• Write-back scheme (dirty bit)

CS385 – Computer Architecture, Lecture 22

Lecture Notes

1. Optimizing address translation - Translation Lookaside Buffer (TLB):
• TLB miss
• Page fault
• TLB associativity
2. MIPS R2000 (DECStation 3100) TLB
3. Overall operation of a memory hierarchy
4. Memory protection with VM
5. Using exceptions for handling TLB misses and pages faults: using EPC and Cause registers
6. Summary of VM

CS385 – Computer Architecture, Lecture 23

Lecture Notes

1. Associativity schemes
2. Placing blocks
3. Miss rates and cache sizes
4. Finding blocks
5. Why do we use full associativity and a separate lookup table (page table) in VM
6. Choosing a block to replace
7. Writing blocks
8. The sources of misses
9. The challenge: reducing the miss rate has a negative effect on the overall performance
10. Pentium Pro and PowerPC 604

Exercises

CS385 – Computer Architecture, Lecture 24

Lecture Notes

1. Buses: lines, transactions, types
2. Synchronous and asynchronous buses
3. Handshaking protocol
4. Bus access: master and slave
5. Bus arbitration schemes
6. Bus standards

CS385 – Computer Architecture, Lecture 25

Lecture Notes

1. The role of the operating system in interfacing I/O devices to Memory
2. Controlling the I/O devices
• Memory mapped I/O
• Special I/O instructions
3. Communicating with the processor
• Polling
• Interrupt-driven I/O
4. Direct memory access (DMA)
5. DMA and the memory system
6. Designing an I/O system: latency and bandwidth constraints.

CS385 – Computer Architecture, Lecture 26

Lecture Notes

1. Basic approaches to sharing data and types of connectivity
2. Programming multiprocessors
3. Multiprocessors connected by a single bus
4. A parallel program
5. Multiprocessor cache coherency
6. Implementing a multiprocessor cache coherency protocol
7. Synchronization using coherency

CS385 – Computer Architecture, Lecture 27

Lecture Notes

1. Shared memory vs. multiple private memories
2. Centralized memory vs. distributed memory
3. Parallel programming by message passing
4. Distributed memory communication
5. Memory allocation
6. Clusters and network topology
7. Modern clusters:
• The Beowulf Project
• AppleSeed Project

CS385 – Computer Architecture, Lecture 28

Lecture Notes

1. Measuring computer performance:
• Program execution time
• CPU time: user CPU time and system CPU time
2. Evaluating computer systems:
• Relative performance
• Workload
• Benchmarks
3. Categories of parallelism
• Single instruction stream, single data stream (SISD)
• Single instruction stream, multiple data streams (SIMD)
• Multiple instruction streams, single data stream (MISD)
• Multiple instruction streams, multiple data streams (MIMD)

CS385 Assignment 1: Assembly Programming in MIPS (maximal grade 5 points)

1. At least one instruction from each instruction type: R-type arithmetic, I-type arithmetic, Memory transfer and Branch.
2. Input and Output through system calls.
3. Comments explaining the type, format and the meaning of each instruction. If you use a pseudo instruction, there must be an explanation how it translates into real MIPS instructions.

Documentation and submission: Submit the source text of the program (ASCII text) as an attachment through Blackboard Vista Courses > CS-385 > Assignment 1.

CS385 Semester Project: Building a mini MIPS machine (total grade 45 points, including 3 progress reports by 10 points each)

CS385 Midterm Test (maximal grade 25 points)

CS385 Final Exam (maximal grade 25 points)