CS462 - Artificial Intelligence

Spring-2005

Classes: TR 5:15 pm - 6:30 pm, RVAC 107
Instructor: Dr. Zdravko Markov, MS 203, (860)-832-2711, http://www.cs.ccsu.edu/~markov/, e-mail: markovz@ccsu.edu
Office hours: MW: 6:45 pm - 7:45 pm, TR: 10:00 am - 12:00 pm, or by appointment

Catalog  Description

Artificial Intelligence ~ Spring. ~ [c] ~ Prereq.: CS 253 or (for graduates) CS 501. ~ Presentation of artificial intelligence as a coherent body of ideas and methods to acquaint the student with the classic programs in the field and their underlying theory. Students will explore this through problem-solving paradigms, logic and theorem proving, language and image understanding, search and control methods, and learning.

Course Goals

Definitions of AI

Required Text

Recommended Text

Required software

General Web Resources

Grading and grading policy

The final grade will be based on: More information about semester project is available here. Descriptions of the exams and the programming assignments are included in the "Tentative schedule of classes and assignments". The paper is described here.

The letter grades will be calculated according to the following table:
 
A A- B+ B B- C+ C C- D+ D D- F
95-100 90-94 87-89 84-86 80-83 77-79 74-76 70-73 67-69 64-66 60-63 0-59

Late assignments will be marked one letter grade down for each two classes they are late. It is expected that all students will conduct themselves in an honest manner and NEVER claim work which is not their own. Violating this policy will result in a substantial grade penalty or a final grade of F.

Tentative schedule of classes and assignments (will be updated regularly)

  1. Introduction - Intelligent Agents
  2. Problem Solving by Search, Introduction to Prolog
  3. Heuristic (Informed) Search
  4. Constraint Satisfaction
  5. Assignment #1: Problem Solving by Searching. Due on February 24.
  6. Games
  7. Knowledge-Based Agents - Propositional and First-Order Logic
  8. Inference in First-Order Logic, Logic Programming and Prolog
  9. Knowledge Representation
  10. Assignment #2: Reasoning with Propositional and First-Order Logic. Due on April 5.
  11. Planning
  12. Uncertainty and Probabilistic Reasoning
  13. Machine Learning - Basic Concepts, Version Space, Decision Trees
  14. Machine Learning - Numeric Approaches, Clustering, Evaluation
  15. Learning with Background Knowledge - Explanation-Based Learning, Inductive Logic Programming
  16. Assignment #3: Probabilistic Reasoning and Learning
  17. Natural Language Processing
  18. Other Topics and Philosophical Foundations

Semester Project

The semester project is based on a suite of projects developed in the framework of a grant from the National Science Foundation (NSF CCLI-A&I Award Number 0409497), "Machine Learning Laboratory Experiences for Introducing Undergraduates to Artificial Intelligence". The student projects done for this course will be an important step in the evaluation of the NSF grant.

To do the semester projects students have to form teams of 3 people (2-people teams should consult the instructor first). Each team chooses one project to work on. The projects to choose from are the following:

  1. Web Document Classification Project
  2. Intelligent Web Browser Project
  3. Character Recognition and Learning with Neural Networks
  4. Clue Deduction: an introduction to satisfiability reasoning
  5. Solving the N-Puzzle Problem
The descriptions of all four projects are available from the WebCT course shell (Campus Pipeline/My Courses).

To complete the project students are required to:

  1. Choose a project and submit the project title and the names of the students on the project team. Do this no later than February 15. Note that there is a restriction that no more than two teams can work on the same project. Projects will be assigned on a first come first serve basis.
  2. Write an initial project description based on the general description provided in the WebCT course. This must include the following (may be discussed with the instructor before submission):
  3. Submit reports on:
  4. Make a presentation of the final report (during the final week).
Requirements for the final project report: The final report must include the following:
  1. General introduction to the area
  2. Description of the problems addressed (experimented with or solved)
  3. Descriptions of the approaches and algorithms used to solve the problems
  4. Descriptions of the software applications used or the programs implemented
  5. Description of the experiments done for each problem attempted or solved
  6. Comment on the relation of the approaches used in the project to the areas of machine learning (ML), search and knowledge representation and reasoning (KR&R). In particular, the following questions should be answered:
Grading: The project grading will be based on the project results and on the completeness (see the requirements), comprehensibility and quality of presentation of the project reports. All students on one project team will get the same grade.

Paper/Presentation

Write a paper (no more than 5 pages) describing an AI area, topic or application not covered in class. Submit the paper by e-mail through the WebCT course template by May 19.

Format of the term paper

Write the paper following the general format of a scientific paper: title, author (individual authors only, no teamwork), abstract, introduction (short description of the subject, goals, structure of the paper), main text  (structured into sections and covering the AI area or topic), conclusion and references. Include all materials you use (including WEB resources) in the reference section and cite them properly in the text. You may include illustrative material (graphs, charts, diagrams, pictures). Type and print the paper using a word processing system (such as Microsoft Word). An example of a scientific paper can be found here.

Policy on copying

The paper must be original, i.e. entirely written by yourself. Since the paper is a survey of an existing area or topic obviously you have to use other materials (books, reference materials, web sources etc.). Every time you use such materials you have to include a citation and put the title and authors of this material in the reference section. Then you can rephrase or even copy a part of the material (provided that you put it in quotes). If you describe general ideas and approaches you have also to cite their authors. Papers that include copied paragraphs and phrases without acknowledgment to their authors will get a grade of F. For more information see the CCSU Policy on Academic Misconduct.

Term paper grading

The term paper grade will be based on the paper format (according to the above mentioned structure), comprehensibility and completeness of the material (including all major issues within the chosen area or topic).

Extra credit

An extra credit of maximum 5 points will be given for the preparation of presentation slides (e.g. in PowerPoint format) and presenting the paper in class. The presentation will last 10-15 minutes and should be arranged with the instructor ahead of time. The presentations should be done before the final week.

Introduction to AI,  Intelligent Agents

1. Goals

2. Definitions of AI

3. Approaches

  1. Models of human reasoning - cognitive modeling (cognitive science), GPS (Newell & Simon, 1972).
  2. Models of human behavior - The Turing Test (1950), Loebner Prize. Natural language processing, knowledge representation, reasoning and learning.
  3. Models of rational thought (logical approach). Aristotle's syllogistic logic: "Socrates is a human, humans are mortal, thus Socrates is mortal". Requires 100% accurate knowledge.
  4. Models of rational behavior - rational agents. Includes both (2) and (3), but is more general.

4. Related Areas

5. History of AI

6. Intelligent Agents

7. References

Problem Solving by Search

1. Problem solving agents

The agent's "world" includes the agent itself and the environment it interacts with. To reach its goal the agent should perform a sequence of actions so that the world reaches the goal state. For this purpose a formal representation of the agent's world is needed:

2. Problem representation

3. Problem

4. Examples

5. Solution

6. Uninformed Search

Heuristic (Informed) Search

1. Why heuristic search

2. Algorithms for heuristic search

3. Properties of A*

4. Heuristic functions

Nearest Neghbor Learning

  1. Distance or similarity function defines what's learned.
  2. Example: weather data

      3.  ID  2  8  9   11
       D(X, ID)  1  2  2  2
       play  no  no yes  yes
  3. Discussion

Naive Bayes

  1. Basic assumptions
  2. Probabilities of weather data

    outlook temp humidity windy play
    sunny hot high false no
    sunny hot high true no
    overcast hot high false yes
    rainy mild high false yes
    rainy cool normal false yes
    rainy cool normal true no
    overcast cool normal true yes
    sunny mild high false no
    sunny cool normal false yes
    rainy mild normal false yes
    sunny mild normal true yes
    overcast mild high true yes
    overcast hot normal false yes
    rainy mild high true no
  3. Bayes theorem (Bayes rule)
  4. Bayes for classification
  5. The “zero-frequency problem”
  6. Missing values
  7. Numeric attributes
  8. Discussion

Assignment #1: Problem Solving by Searching (max grade 10 pts.)

Do the following and submit the results by February 24
  1. Represent the problem of sorting a four-element list as a state space search problem:
  2. Show how the problem is solved in both cases by simple 3-4 step hand solved examples.
  3. Write two Prolog programs that represents both types of state spaces.
  4. Implement an admissible heuristic function based on the number of pairs of neighboring elements that are not in the right order. Add the function to the representation (define a rule for h(Node, Value) and include it in the files with the state transitions). Explain what makes the function admissible and how it can be made inadmissible.
  5. Sort the list (4,3,2,1) (goal state: (1,2,3,4)) by  using all uninformed (search1.pl) and heuristic (search2.pl) search algorithms: depth_first, breadth_first, iterative_deepening, uni_cost, best_first, a_star and beam (with n=10 and n=1) and collect statistics about their performance. Do this for both types of state spaces.
  6. Compare the performance of all those algorithms by the following criteria: time complexity (number of explored nodes), space complexity (max length of the queue) and optimality (put the results in a table and explain the reasons for getting each best and worst result for each performance criterion). Again do this for both types of state spaces (use two separate tables).
  7. Compare now the two types of state representations. Which space is bigger? Why? How does this affect the search algorithms' perfomance.
Extra credit (max 2 pts.): Represent the 4-element sorting problem as a  constraint satisfaction task and solve it by using csp.pl. Explain the representation and compare it with the state space search representation.

Documentation and submission: Write a report describing the solutions to all problems and answers to all questions and mail it as an attachment  to my instructors account for the WebCT (available through Campus Pipeline/My Courses/Artificial Intelligence).

Assignment #2: Reasoning with Propositional and First-Order Logic (max grade 10 pts. + 5 pts. extra credit)

Do the following and submit the results by April 5:
  1. Use the wumpus world shown in Figure1 of logic.pdf
  2. Represent the upper left corner of the wumpus world (rooms: (3,1), (3,2), (4,1), (4,2)). For each room encode the knowledge about perceiving stench and breeze in the rooms neighboring to the beast and pit. Restriction: put the perceptions in the "if" part of the implications (left of "->").
  3. Create two versions of the representation - one in PL and one in FOL.  Then for each language prove the presence of the beast in room (3,1) given the agent perception in rooms (3,2), (4,1) and (4,2). Do this by:
  4. Define the complete wumpus world (4x4 board) in FOL and translate it into clausal form (use wumpus_fol from logic.pl):
Extra credit (max 5 pts.): Program an agent for the wumpus world that starts at room (1,1) facing east and visits all logically provable safe rooms. Restriction: action rules cannot have more than two actions. Explain the agent's reasoning and actions and test it with wumpus.pl.

Documentation and submission: Write a report describing the solutions to all problems and answers to all questions and mail it as an attachment  to my instructors account for the WebCT (available through Campus Pipeline/My Courses/Artificial Intelligence).

Assignment #3: Probabilistic Reasoning and Learning (max grade 20 pts. + 5 pts. extra credit)

Due date: May 10

Use the weather (tennis) data in tennis.pl and do the following:
  1. Create a Bayesian network for the weather data using the approach taken in loandata.pl. Then use this network with bn.pl to:
  2. Add taxonomies for the attributes so that they can be used as structural (see how this is done in loandata.pl). Then use version space learning (vs.pl) to create the largest possible concepts for each class ("yes" and "no") by putting an example from it in the beginning of the set of examples. In other words, find a good order of the examples (one starting with an example from class "yes" and one - from class "no"), so that the program converges after reading as many as possible examples. Use the following approach (see also Version space learning):
  3. Use decision tree learning (id3.pl) with the weather data and:
  4. Use Naive Bayes (bayes.pl) and Nearest Neighbor (knn.pl with k=1,3,5 and knnw with k=14) and:
Extra credit (max 5 pts.): Use agglomerative clustering (cluster.pl with min and max parameter) and:

Documentation and submission: Write a report describing the solutions to all problems  and mail it as an attachment  to my instructors account for the WebCT (available through Campus Pipeline/My Courses/Artificial Intelligence).

Midterm Test (max grade 10 pts.)

The Midterm test will be available in the WebCT course template from April 1 through April 4. There will be 10 multiple choice or short answer questions that will have to be answered within 2 hours.

The test includes the following topics:

Final Test (max grade 10 pts.)

The Final test will be available in the WebCT course template from May 13 through May 19. There will be 10 multiple choice or short answer questions that will have to be answered within 2 hours.

The test includes the following topics:

Last updated: 5-10-2005