Time & Place:	Spring 2024, was TuTh 12:30-1:45, 107 Patterson Office Tower but now meets TuTH 12:15-1:30 108 Marksbury
Course URL:	http://aggregate.org/PCIOT/
Instructor:	Professor Hank Dietz, http://aggregate.org/hankd/, research at http://aggregate.org/
Audience:	This course is suitable for both Undergrad & Graduate students; especially Computer Engineering/CS/EE majors, but also visual artists.
Prerequisites:	Some familiarity with C, C++, or Arduino programming. No photography background is required.
Equipment used:	Students will be loaned Canon PowerShot cameras for the duration of the course and will be given ESP32-CAM hardware. Ideally, each student should have a laptop that can support Arduino and CHDK programming.
Programming languages used:	C/C++/Ardunio and some CHDK Lua.
Class meetings:	Class meetings generally will be in the assigned classroom, but some class meetings may instead be held either outdoors (e.g., capturing images) or as a "virtual flipped classroom" using Zoom to facilitate interactive sharing/debugging of code being written for the projects. There also are potentially useful facilities in the 108A Marksbury research lab.
Graduate vs. Undergraduate	Graduate students should register for EE699-001 and will have enhanced projects as compared to Undergraduate students registered for EE599-001. Course content is identical.
Syllabus:	PDF

This is essentially a project-intensive embedded systems course about using digital cameras.

This course will start by introducing the basic principles of photography and the details of how digital cameras work. However, cameras are no longer just about photography; they are sensors in embedded computing systems that can serve a wide range of applications. For example, using CHDK, it is trivial to program a Canon PowerShot camera to serve as a non-contact tape measure. The course will use CHDK cameras, such as the SX530 HS (shown above), to explain how camera internals work and students will get hands-on experience using and programming these cameras. Cameras are also now cheap sensors for use within Internet of Things (IoT) devices. An ESP32-CAM IoT module (also shown above) that costs under $10 includes a 2MP camera and can be programmed for tasks as diverse as wirelessly serving live video via an HTML browser interface to unlocking a door when a person's face is recognized. We will discuss IoT devices in general and use of the ESP32-CAM and its OV2640 camera in particular. Students will implement simple IoT projects using the ESP32-CAM via the Arduino programming environment.

References

Course overview slides

Introduction to Digital Photography

CameraLabs Canon PowerShot SX530 HS review

10 Top Photography Composition Rules, which I mosty agree with ;-)
(a few shots to critique: 0, 1, and 2)

Image editing tools

• GIMP (GNU Image Manipulation Program), sort-of one tool to do everything -- like photoshop
• RawTherapee, a fairly sophisticated raw image file converter to produce JPEGs, etc.
• ImageMagick, a C library for displaying, converting, and editing raster and vector image files; the demo applications make great command-line editing tools
• FFmpeg, the complete, cross-platform solution to record, convert and stream audio and video (this is one of the things started by Fabrice Bellard)

CHDK (Canon Hack Development Kit)

• The User Manual
• CHDK for Computational Photography page at Aggregate.Org
• It is worth noting that using the camera via USB to mount the SD card does bypass the write protection, but mounts the DCIM directory only
• CHDKPTP, which actually works rather well... provided there isn't anything else grabbing PTP devices (there often is, in which case it operates V-E-R-Y S-L-O-W-L-Y; the culprit is usually gphoto and/or gvfs)

In Search of Sensors is a little overview of what's around the sensor in a digital camera
Related: Shuttering methods and the artifacts they produce

Ghost in the Machine is a little overview of the computational environment inside a digital camera

Stuff you didn't know about Lenses is a little overview of camera lens properties and design

ESP32-CAM (especially AI-Thinker version)

• We use the Arduino IDE, which can be installed on Windows, Mac OS X, or Linux as outlined here:
  1. Install Arduino environment
  2. Add https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_index.json to Arduino IDE "Additional Board Manager URLs"
  3. In Arduino Board Manager, install "esp32 by Espressif Systems"
  4. Allow Arduino IDE to access /dev USB ports under Linux by "sudo usermod -a -G dialout user" where user is your login name; you need to log out and login again to have it take effect
  5. You need PySerial installed (which probably means you need pip), and on Linux you might need to install python-is-python3
• An excellent first program to run on the ESP32-CAM is CameraWebServer. You will need to edit the ssid and password to match your local network. There are many variants of this program around... for example, this
• ESP32-CAM for Computational Photography page at Aggregate.Org gives some key pointers

Cambridge in Colour is "a learning community for photographers" with some excellent tutorials including a variety of interactive displays and calculators

Course Staff

Professor Hank Dietz would normally be in the Davis Marksbury Building; see http://aggregate.org/hankd/ for complete contact info. He has an "open-door" policy that whenever his door is open and he's not busy with someone else, he's available -- and yup, there really is a slow-update live camera in his 203 Marksbury office so you can check. During the pandemic, and you should wear a mask if meeting in his office. The best method to contact him is to email hankd@engr.uky.edu using "PCIOT" in the subject line for anything related to this course. If appropriate, individual Zoom meetings also can be scheduled via email.

About the instructor: Professor Dietz is known primarily for his work in supercomputing, but has a long history in photography. He was photo editor for his high school newspaper and yearbook. As an undergrad, he worked on Columbia University's Spectator newspaper and Broadway magazine. He has had news photos published in the New York Times and commercial photography published in the Saturday Evening Post. Photography faded into a hobby as he became known for his computer engineering research... until 1996, when he built a 30MP video wall for a cluster supercomputer. Since then, he has been doing all sorts of work treating cameras as computing systems, trying to improve image quality and give cameras new abilities. See http://aggregate.org/DIT/ for an overview of his research and publications in digital imaging technologies.