CS385 - Computer Architecture

Spring-2024

Catalog description: The architecture of the computer is explored by studying its various levels: physical level, operating-system level, conventional machine level and higher levels. An introduction to microprogramming and computer networking is provided.

Course Prerequisites: CS 354

Prerequisites by topic

• Basic skills in software design and programming
• Assembly language programming and basics of computer organization
• Digital systems design
• Boolean algebra and discrete mathematics

Course description

1. MIPS instruction set
2. Computer arithmetic and ALU design
3. Datapath and control
4. Using Hardware Description Language to design and simulate the CPU
5. Pipelining
6. Memory hierarchy, caches and virtual memory
7. Interfacing CPU and peripherals, buses
8. Multiprocessors, networks of multiprocessors, parallel programming
9. Performance issues

Course Learning Outcomes (CLO)

1. Understand the fundamentals of different instruction set architectures and their relationship to the CPU design.
2. Understand the principles and the implementation of computer arithmetic.
3. Understand the operation of modern CPUs including pipelining, memory systems and busses.
4. Understand the principles of operation of multiprocessor systems and parallel programming.
5. Design and emulate a single cycle or pipelined CPU by given specifications using Hardware Description Language (HDL).
6. Work in teams to design and implement CPUs.
7. Write reports and make presentations of computer architecture projects.

• SO-2: Design, implement, and evaluate a computing-based solution to meet a given set of computing requirements in the context of the program s discipline (supported by CLO's 5, 6, 7).
• SO-6: Apply computer science theory and software development fundamentals to produce computing-based solutions (supported by CLO's 1, 2, 3, 4).

Required textbook

• Patterson and Hennessy, Computer Organization and Design MIPS Edition: The Hardware/Software Interface, 6th Edition, Elsevier 2021, ISBN: 9780128201091.

Required software

1. Icarus Verilog: HDL compiler and simulator available for download from http://bleyer.org/icarus/ and online at https://www.jdoodle.com/execute-verilog-online.
2. SPIM simulator: A software simulator for running MIPS32 programs available for Windows and other platforms.

Class Participation: Active participation in class is expected of all students. Regular attendance is also expected. If you must miss a class, try to inform the instructor of this in advance. In case of missed classes and work due to plausible reasons (such as illness or accidents) limitted assistance will be offered. Unexcused absences will result in the student being totally responsible for the make-up process.

Course Expectations for Out-of-Class Work: To succeed in this 3-credit class, it is expected that you commit a total of 12 hours per week to master the course material. This includes 2.5 hours of lecture time and an additional 9.5 hours dedicated to independent study and coursework. This time commitment aligns with the expectations set by the Computer Science department for major courses and adheres to university policies. Recognizing that dedicating this amount of time outside the classroom is a significant commitment, it is nevertheless necessary for success. Please plan your course load accordingly.

Honesty policy: The CCSU honor code for Academic Integrity is in effect in this class. It is expected that all students will conduct themselves in an honest manner and NEVER claim work which is not their own. Violating this policy will result in a substantial grade penalty, and may lead to expulsion from the University. You may find it online at http://web.ccsu.edu/academicintegrity/. Please read it carefully.

Grading: Grading will be based on one programming assignment (10%), a midterm test (20%), a final exam (25%) and a semester project (45%, including progress reports, the final documentation, and the presentation). The letter grades will be calculated according to the following table:
 

A	A-	B+	B	B-	C+	C	C-	D+	D	D-	F
95-100	90-94	87-89	84-86	80-83	77-79	74-76	70-73	67-69	64-66	60-63	0-59

Unexcused late submission policy: Submissions made more than two days after the due date will be graded one letter grade down. Submissions made more than a week late will receive two letter grades down. No submissions will be accepted more than two weeks after the due date.

Students with disabilities: Students who believe they need course accommodations based on the impact of a disability, medical condition, or emergency should contact me privately to discuss their specific needs. I will need a copy of the accommodation letter from Accessibility Services in order to arrange class accommodations. Contact Office of Accessibility Services, Willard-DiLoreto Hall, Suite W201 if you are not already registered with them. Office of Accessibility Services maintains the confidential documentation of your disability and assists you in coordinating reasonable accommodations with your faculty.

Tentative schedule of classes and assignments

1. Jan 18: Introduction: Computer Architecture = Instruction Set Architecture + Machine Organization
2. Jan 23: Review of HDL (Figure 6.5, behavioral_serial_adder.vl, Digital Design Review Assignment)
3. Jan 25: MIPS Instructions: arithmetic, registers, memory, fecth&execute cycle
4. Jan 30: MIPS Instructions: control and addressing modes
5. Feb 1: Computer arithmetic and ALU design: representing numbers, arithmetic and logic operations
6. Feb 1: Submit Digital Design Review Assignment (optional, for extra credit)
7. Feb 6: ALU design: multiplication, representing floating point numbers
8. Feb 8: ALU design: full adder, slt operation, HDL design
9. Feb 13: Class canceled due to snow storm
10. Feb 13: Assignment 1 due (10 pts.)
11. Feb 15: The Processor: Building a datapath
12. Feb 15: Semester Project is posted. Form teams for the Semester Project. Email me the team members.
13. Feb 20: The Processor: Control (single cycle approach)
14. Feb 22: Using Hardware Description Language to Design and Simulate the MIPS processor. Review of Semester Project Report #1.
15. Feb 27: Introduction to pipelining
16. Feb 29: Progress Report #1 due (10 pts.): A simpilfied single-cycle datapath capable of executing immediate and R-type instructions. See Semester Project for details.
17. Feb 29: Solving pipeline hazards
18. March 5: Implementing pipeline datapath and control
19. March 7: Implementing data and branch hazards control
20. March 19: Review of Datapath, Control and Pipelining. Review of Progress Report 2.
21. March 21: Review for Midterm Test
22. March 26: Midterm Test (20 pts.)
23. March 26: Progress Report #2 due (10 pts.): Complete single-cycle datapath. See Semester Project for details.
24. March 28: Implementing a 3-stage pipeline in HDL (mips-pipe3.png, mips-pipe3.vl). Progress Report #3 posted.
25. April 2: Memory hierarchy, The Basics of caches
26. April 4: Improving cache performance
27. April 9: Virtual Memory
28. April 11: Virtual Memory optimization
29. April 11: Progress Report #3 due (10 pts.): 3-stage pipelined datapath for immediate and R-type instructions. See Semester Project for details.
30. April 16: Review of Final Report (complete 5-stage pipeline, mips-pipe.vl)
31. April 18: A general framework of memory hierarchies
32. April 23: Multiprocessors
33. April 25: Networks of muiltiprocessors
34. April 30: Final Project Report and Presentation Slides due (10 pts.)
35. April 30: Semester Project Presentations, Review for Final Exam
36. May 2: Semester Project Presentations, Review for Final Exam
37. May 7, 8:00 AM - 10:00 AM Final Exam (25 pts.)

CS385 Computer Architecture, Lecture 1

Lecture Notes

1. Levels of Abstraction
2. Computer Architecture = Instruction Set Architecture + Machine Organization
3. Instruction Set The Software Hardware Interface
4. Levels of Computer Architecture in More Depth
• Software:
• Application
• Operating System
• Firmware
• Instruction Set Architecture:
• Organization of Programmable Storage
• Data type and Structures: encodings and machine representation
• Instruction set
• Instruction Formats
• Addressing Modes and Accessing Data and Instructions
• Exception Handling
• Hardware:
• Instruction Set Processing
• I/O System
• Digital Design
• Circuit Design
• Layout
5. Basic Components of a Computer
• Processor: Datapath and Control
• Memory
• I/O
6. Computer Organization
• Capabilities and Performance of the Basic Functional Units
• The Way These Units are Interconnected
• Information Flow between components
• Information Flow Control
7. Performance
• Measuring and improving computer performance
• Program execution time
• CPU time
• Power
• Evaluating computer systems
• Relative performance
• Workload
• Benchmarks
• Amdahl's Law

CS385 Computer Architecture, Lecture 2

Lecture Notes

1. Design goal: maximize performance and minimize cost. Primitive (low level) and very restrictive instructions (fixed number and type of operands).
2. Design principles:
• Simplicity favors regularity (uniform instruction format)
• Smaller is faster (only 32 registers)
• Good design demands a compromise (I-type instructions)
3. MIPS arithmetic: 3 operands, fixed order, registers only.
4. Using only registers: R-type instructions.
5. Registers: 32-bits long, conventions.
6. Memory organization: words and byte addressing.
7. Data transfer (load and store) instructions. Example: accessing array elements.
8. Translating C code into MIPS instructions the swap example.
9. Machine Language: instruction format, I-type (Immediate) format for data transfer
10. Stored program concept (Von Neumann Architecture): programs in memory, fetch & execute cycle

CS385 Computer Architecture, Lecture 3

Lecture Notes

1. Implementing the C code for if in MIPS: conditional branch.
2. Implementing the C code for if else in MIPS: unconditional branch
3. Simple for loop
4. Check for less-than: building a pseudoinstuction for branch if less-than.
5. Addressing in branch instructions: PC-relative and pseudodirect.
6. Constants: use of immediate addressing (constants as operands addi, slti, andi, ori)).
7. 32-bit constants manipulate upper 2 bytes separately (load upper immediate)
8. Summary of MIPS addressing: register (add), immediate (addi), base or displacement (lw), PC-relative (bne), pseudodirect (j)

CS385 Computer Architecture, Lecture 4

Lecture Notes

1. Representing numbers: sign bit, one's complement, two's complement.
2. Arithmetic: addition, subtraction, detecting overflow.
3. Logical operations: shift, and, or.
4. Basic ALU building components: and-gate, or-gate, inverter, multiplexor.
5. ALU for logical operations.
6. ALU for add, and, or.
7. Supporting subtraction

Tutorials and practice quizzes on two s complement numbers

• http://chortle.ccsu.edu/AssemblyTutorial/Chapter-08/ass08_1.html (tutorial)
• http://chortle.ccsu.edu/AssemblyTutorial/Chapter-08/ass08quiz.html (quiz)

CS385 Computer Architecture, Lecture 5

Lecture Notes

1. Implementation of a full adder:
• Carry out logic
• Result logic: using 'and', 'or' and inverter and using xor-gate.
2. Supporting set on less-than (slt).
3. Test for equality (needed for branching)
4. Designing the ALU in Verilog
5. Carry Lookahead

CS385 Computer Architecture, Lecture 6

Lecture Notes

1. Implementing multiplication:
• Using 64-bit adder;
• Using 32-bit adder for the upper 32-bit of the product;
• Avoiding the use of the multiplier register.
2. Floating point numbers
• Scientific notation: (-1)^sign * significand * 2^exponent
• Range and precision (overflow and underflow).
• IEEE 754 floating point standard - allows integer comparison:
• normalized representation
• implicit leading 1
• exponent is biased: exponent in [0..0 (most negative), 1..1 (most positive)]
• bits of the significand represent the fraction between 0 and 1.
• (-1)^S * (1 + s1*2^-1 + s2*2^-2 + ...) * 2^(exponent-bias)
• Problems with floating point arithmetic

• http://chortle.ccsu.edu/AssemblyTutorial/Chapter-30/ass30_1.html (tutorial)
• http://chortle.ccsu.edu/AssemblyTutorial/Chapter-30/ass30quiz.html (quiz)

CS385 Computer Architecture, Lecture 7

Lecture Notes

1. Abstract level implementation:
• Instruction memory
• Program counter
• Register file
• ALU
• Data memory
2. Basic building elements
• Combinational logic
• State elements: D-lathes and D flip-flops
• Clocking methodology: edge triggered
3. Fetching instructions and incrementing the program counter
4. Register file and execution of R-type instructions
5. Datapath for lw and sw instructions (add data memory and sign extend)
6. Datapath for branch instructions

CS385 Computer Architecture, Lecture 8

Lecture Notes

1. ALU control: mapping the opcode and function bits to the ALU control inputs
2. Designing the main control unit
3. Operation of the Datapath (single-cycle implementation):
• R-type instructions
• Load (store) word
• Branching instructions
4. Problems of the single-cycle implementation

CS385 Computer Architecture, Lecture 11

• Simplified MIPS for R-types and addi: Diagram, HDL implementation (mips-r-type_addi.vl)
• MIPS single cycle implementation: Diagram, HDL implementation (mips-simple.vl)

CS385 Computer Architecture, Lecture 13

Lecture Notes

1. Pipelining by analogy (laundry example):
• Pipelining helps throughput of the entire workload
• Multiple tasks operating simultaneously and using different resources
• Potential speedup = number of pipe stages
• The pipeline rate is limited by the slowest stage
• Unbalanced lengths of pipe stages reduces the speedup
• The time to "fill" the pipeline and the time "drain" it reduces the speedup
• Stall for dependencies
2. Five stages of the load MIPS instruction
3. The pipelined datapath
4. Single cycle, multiple cycle vs. pipeline
5. Advantages of pipelined execution
6. Problems with pipelining (pipeline hazards)
• Structural hazards
• Data hazards
• Control hazards

CS385 Computer Architecture, Lecture 14

Lecture Notes

1. Structural hazards: single memory
2. Control hazards:
• Stall: wait until decision is clear
• Predict: fixed prediction (e.g. fail), dynamic prediction (based on history)
• Delayed brach (software solution):
add $4, $5, $6            beq $1, $2, $40
beq $1, $2, 40     ==>    add $4, $5, $6
lw $3, 300($0)            lw $3, 300($0)
3. Data hazards (dependecies backwards in time):
• Forwarding (bypassing)
• Reordering code
lw $t0, 0($t1)               lw $t0, 0($t1)
lw $t2, 4($t1)      ==>      lw $t2, 4($t1)
sw $t2, 0($t1)               sw $t0, 4($t1)
sw $t0, 4($t1)               sw $t2, 0($t1)
4. Designing a pipelined processor

CS385 Computer Architecture, Lecture 15

Lecture Notes

1. Splitting datapath into stages: using registers to store parts of the instruction
2. Transferring data forward and backward between the stages: lw example
3. Corrected datapath: storing rd for the write back stage.
4. Graphically representing pipelines: multiple-clock-cycle vs. single-clock-cycle diagram
5. Pipeline control:
• IF: no control signals to store for later stages (they are always asserted).
• ID: no control signals to store for later stages (they are always asserted).
• EX: set RegDst, ALUOp, ALUSrc
• MEM: set Branch, MemRead, MemWrite
• WB: set MemtoReg, RegWrite
6. Datapath with control
7. Example: running this code through the pipeline in 9 cycles.

lw   $10, 20($1)
sub  $11, $2, $3
and  $12, $4, $5
or   $13, $6, $7,
add  $14, $8, $9

CS385 Computer Architecture, Lecture 16

Lecture Notes

1. Detecting data dependencies
• EX/MEM.Rd = ID/EX.Rs
EX/MEM.Rd = ID/EX.Rt
• MEM/WB.Rd = ID/EX.Rs
MEM/WB.Rd = ID/EX.Rt
2. Forwarding
• if (EX/MEM.RegWrite and EX/MEM.Rd = ID/EX.Rs)ForwardA = 10
if (EX/MEM.RegWrite and EX/MEM.Rd = ID/EX.Rt)ForwardB = 10
• if (MEM/WB.RegWrite and MEM/WB.Rd = ID/EX.Rs)ForwardA = 01
if (MEM/WB.RegWrite and MEM/WB.Rd = ID/EX.Rt)ForwardB = 01
3. Data hazards and stalls
If (ID/EX.MemRead and
    (ID/EX.Rt = IF/ID.Rs or
     ID/EX.Rt = IF/ID.Rt))
   stall the pipeline
4. Branch hazards
• Reducing the delay of branches - move up the address calculation (move the adder) and the branch decision (add XOR and AND gates)
• Assuming the branch will not be taken
• Flashing instructions in IF, ID, and EX stages, if the branch is taken
5. Advanced pipelining
• Superpipelining
• Superscalar
• Dynamic scheduling

CS385 Computer Architecture, Lecture 17

Lecture slides (PDF)

Lecture Notes

Datapath

1. Abstract level implementation:
• Instruction memory
• Program counter
• Register file
• ALU
• Data memory
2. Basic building elements
• Combinational logic
• State elements: D-lathes and D flip-flops
• Clocking methodology: edge triggered
3. Basic operations
• Instructions fetch
• Accessing register file and execution of R-type instructions
• Datapath for lw and sw instructions (add data memory and sign extend)
• Datapath for branch instructions

Control

1. ALU control: mapping the opcode and function bits to the ALU control inputs
2. Designing the main control unit
3. Operation of the Datapath (single-cycle implementation):
• R-type instructions
• Load (store) word
• Branching instructions
4. Problems of the single-cycle implementation

Pipelining

1. Basic principles of pipelining
• Pipelining helps throughput of the entire workload
• Multiple tasks operating simultaneously and using different resources
• Potential speedup = number of pipe stages
• The pipeline rate is limited by the slowest stage
• Unbalanced lengths of pipe stages reduces the speedup
• The time to "fill" the pipeline and the time "drain" it reduces the speedup
• Stall for dependencies
2. MIPS pipelining: the five stages of the lw instruction
3. Problems with pipelining:
• Structural hazards
• Data hazards
• Control hazards
4. Designing a pipelined processor
5. Transferring data forward and backward between the stages
6. Pipeline control
7. Implementing data and branch hazard control
• Detecting data dependencies
• Forwarding
• Data hazards and stalls
• Branch hazards
8. Advanced pipelining

Demo:

• Run wk6animations.ppt in Blackboard Learn
• Section 4.13 (http://booksite.elsevier.com/9780124077263/appendices.php), pages 16-30

CS385 Computer Architecture, Lecture 18

Lecture Notes

1. Memory technologies and trends
2. Impact on performance
3. The need of hierarchical memory organization
4. The principle of locality
5. Memory hierarchy terminology
6. Basics of RAM implementation
• SRAM:  D-latches, three-state buffers, address decoders, two level addressing
• DRAM:  DRAM cell, refreshing
• Error detection and correction

CS385 Computer Architecture, Lecture 19

Lecture Notes

1. Direct-mapped cache
2. Accessing a cache
• Cash index
• Tag
• Valid bit
3. Writing to the cache (write-through and write-back schemes)
4. Handling cache misses
• Read miss: load the word from memory
• Write miss: write both to the cache and to the memory (using write buffer)
5. Spatial locality caches: keeping consistency on write
6. Main memory organization

Exercises: Problems 5.1, 5.2, 5.3, 5.7 from Chapter 5 Exercises in Blackboard

CS385 Computer Architecture, Lecture 20

Lecture Notes

1. Measuring cache performance
• Stall clock cycles = Instructions * Miss rate * Miss penalty
• Example 1 (reducing CPI):
• 2% - instruction miss; 4% - data miss; 36% - lw/sw; 40 cyc - miss penalty.
• 2 CPI => CPI_stall = 3.36, i.e. perfect cache is 1.68 times faster.
• 1 CPI => CPI_stall = 2.36, i.e. perfect cache is 2.36 times faster.
• Example 2: doubling clock rate => 80 cyc - miss penalty, CPI_stall = 4.75, Performance: 3.36/(4.75/2) = 1.41 faster with stalls, 2 times faster without stalls.
• Conclusion: cache penalties increase as the machine becomes faster
2. Flexible placement of blocks in the cache
• Direct mapped: Cache index = (Block address) modulo (Cache size); no search; small tag.
• Set associative: Cache index = (Block address) modulo (Number of sets in cache); search the set; larger tag.
• Fully associative: Cache index is not determined; search the whole cache; tag = address.
3. Locating a block in the cache: N-way cache requires N comparators and N-way multiplexor
4. Choosing which block to replace: least recently used
5. Multilevel caches

• Write a sequence of memory references for which:
• the direct mapped cache performs better than the 2-way associative cache;
• the 2-way associative cache performs better than the fully associative cache.
• Exercises 5.1, 5.2, 5.3, 5.7 from Chapter 5 Exercises in Blackboard

CS385 Computer Architecture, Lecture 21

Lecture Notes

1. The need of VM
• Many programs (processes) can use a single memory
• Use a memory exceeding the size of the main memory
2. VM organization and terminology: virtual address, physical address, page, page offset, page fault, memory mapping (translation).
3. Design decisions motivated by the very high cost of page faults:
• Large pages (4K-64K)
• Reducing page fault penalties: fully associative VM
• Software management of page faults
• Write-back instead of write-through
4. Addressing pages:
• Page table, page table register
• Processes (active, inactive) and page tables
• Page faults
• Replacing pages: LRU, reference (use) bit
• Write-back scheme (dirty bit)

CS385 Computer Architecture, Lecture 22

Lecture Notes

1. Optimizing address translation - Translation Lookaside Buffer (TLB):
• TLB miss
• Page fault
• TLB associativity
2. MIPS R2000 (DECStation 3100) TLB
3. Overall operation of a memory hierarchy
4. Memory protection with VM
5. Using exceptions for handling TLB misses and pages faults: using EPC and Cause registers
6. Summary of VM

CS385 Computer Architecture, Lecture 23

Lecture Notes

1. Associativity schemes
2. Placing blocks
3. Miss rates and cache sizes
4. Finding blocks
5. Why do we use full associativity and a separate lookup table (page table) in VM
6. Choosing a block to replace
7. Writing blocks
8. The sources of misses
9. The challenge: reducing the miss rate has a negative effect on the overall performance
10. Pentium Pro and PowerPC 604

Exercises (from Chapter 5 Exercises in Blackboard)

• Direct-mapped chaches: 5.2.1, 5.2.2, 5.3
• Associative caches: 5.7.1, 5.7.2, 5.7.3, 5.13.1
• Virtual memory: 5.11.1, 5.11.4, TLB example

CS385 Computer Architecture, Lecture 26

Lecture Notes

1. Amdahl's Law
2. Basic approaches to sharing data and types of connectivity
3. Programming multiprocessors
4. Multiprocessors connected by a single bus
5. A parallel program
6. Multiprocessor cache coherency
7. Implementing a multiprocessor cache coherency protocol
8. Synchronization using coherency, locks, atomic swap operation

again: addi $t0, $0, 1 # copy locked value
ll $t1, 0($s1) # load linked
sc $t0, 0($s1) # store conditional
beq $t0, $0, again # branch if store fails
add $s4, $0, $t1 # put load value in $s4

CS385 Computer Architecture, Lecture 27

Lecture Notes

1. Shared memory vs. multiple private memories
2. Centralized memory vs. distributed memory
3. Parallel programming by message passing
4. Distributed memory communication
5. Memory allocation
6. Clusters and network topology
7. Modern clusters:
• AppleSeed Project
• Message Passing Interface (MPI)
• BOINC
• SETI@home
• Folding@home
• List of distributed computing projects from Wikipedia

Digital Design Review Assignment

Assignment 1: Assembly Programming in MIPS (maximum grade 10 points)

Semester Project: Building a mini MIPS machine (maximum grade 45 points including the presentation)

Midterm Test (20 points)

• Number systems: binary, two's complement, floating point, conversions between decimal and two's complement and floating point.
• MIPS instruction set architecture and assembly programming
  • Instruction format and meaning
  • Accessing memory. Note the byte order within the memory word (big-endian, little-endian), see book page A-43
• Single-cycle datapath and control (see http://www.cs.ccsu.edu/~markov/ccsu_courses/385SL8.pdf/slide #5)
• MIPS single-cycle implementation in Verilog HDL (mips-simple.vl)
  • Implementing basic CPU components and control
  • Implementing and addressing instruction and data memory
• Pipelining
• Number of pipeline stages each instruction takes. Note the particular implementation of the branch to reduce the delay on branching (moving branch decision earlier). See book Section 4.8, Fig. 4.65 (http://www.cs.ccsu.edu/~markov/ccsu_courses/385SL16.pdf/last slide).
• Executing code on the pipelined MIPS (sample questions: How many cycles does this code take?, What is the ALU doing in cycle 9?)
• Identifying data dependencies and hazards in the code
• Resolving hazards by changing code (inserting nops or reordering instrutions)

Final Exam (25 points)

• Processor stages and timing
• Single cycle
• Pipelined
• Pipelining
• Forwarding (checking conditions on pipeline registers)
• Hazard detection and stalling
• Solving branch hazards
• Memory System
• Temporal and spatial locality in programs
• Cache hits and misses
• Cache size
• Virtual memory
• Overall operation of a memory hierarchy
• Direct-mapped chaches: 5.2.1, 5.2.2, 5.3
• Associative caches: 5.7.1, 5.7.2, 5.7.3, 5.13.1
• Virtual memory: 5.11.1, 5.11.4, TLB example
• Multiprocessors
• Amdahl's Law
• Multiprocessor architectures
• Processor Synchronization
• Instruction and data streams
• Parallel programming