Home » About » Research » Publications


PhD Thesis Dissertation

Multimodal Analytics for Healthcare, PhD Thesis Dissertation by Carlos Torres. Electrical and Computer Engineering Department (ECE), University of California Santa Barbara (UCSB), Santa Barbara, CA 93106.  [pdf] [Slides][BibTex]


  • Doctors Jeffrey C. Fried* (SBCH MICU) and Scott D. Hammond** (UCSF),
  • UCSB ECE Professors Joao P. Hespanha, Kenneth Rose, and B. S. Manjunath (Chair)

Defended November 20th, 2017; Submitted January 15th, 2018; Conferred January 2018.

* Santa Barbara Cottage Hospital Medical Intensive Care Unit
** University of California San Francisco


Deep Learning & Machine Vision and Learning

Complex Pairwise Activity Analysis Via Instance Level Evolution Reasoning. Sudipta Paul, Carlos Torres, Shivkumar Chandrasekaran, and Amit Roy-Chowdhury.

Overview. Architecture for the collision activity detection system. It will first compute feature embedding for each object appearance of each frame by concatenating the output of bidirectional GRU, two-layer MLP, and temporal reasoning module. This embedding encodes temporal evolution information. Then we use the spatial reasoning module to learn the pairwise interaction. Finally, we compute the activity detection confidence for each pair of objects as well as individual objects of a frame
Sample collision detected by the activity detection module (solid line connecting the colliding cars). During collision, the pairwise activity detection is marked by green line. Thick bounding box around the object represents the unary activity detection.

Uncertainty and Robustness Estimation Methods for Graphical Neural Networks: Towards a Better Understanding of Architecture Performance. Carlos Torres, Tejaswi Nanjudaswamy, and Shivkumar Chandrasekaran.

Uncertainty and Robustness Estimation Methods (UREM) Overview

Sample Random Attack Results

Sample Targeted Attach Results

Smple image manipulated, which fools a neural network. From left to right the images are: (a) original speed boat image that is correctly classified; (b) the cat target-label image; (c) the image with visually imperceptible modifications that is missclassified by AlexNet as a cat with high confidence.

Sample Layer-Wise Targeted Attack.

Results for: AlexNet (left), VGG16 (center), and ResNet-152(right). Blue bars: standard deviation required to change the class; Red bars: probability of the modified detection.

[PrePrint][BibTex] [Datasets: COCO2014 and ImageNet][Code–soon]

Hyperspectral & Mask-RCNN Methods for Small-Pixel Footprint Methane Plume detection

H-mrcnn: Deep Remote Sensing Methods for Methane Detection in Overhead Hyperspectral Imagery. Carlos Torres, Satish Kumar, Alana Ayasse, Robert Dar, and B. S. Manjunath. — pre-print

Figure. From left to right. Overhead view of hyperspectral data with methane plumes as red masks (top row) overlayed on the observed terrain (bottom row). The columns are ground-truth (first column), naive single-band prediction (center column), and H-mrcnn ensemble prediction (right column). The core detector is the mask-rcnn detector with ResNet-101 as backbone.


ACAMs: Actor Conditioned Attention Maps for (Atomic) Action Detection. Oytun Ulutan, Swati Rallapalli, Mudhakar Srivatsa, Carlos Torres, B.S. Manjunath. — ArXiv Pre-Print


HEAL: Healthcare Event and Activity Logging. Carlos Torres, Jeffrey C. Fried, B. S. Manjunath. IEEE-EMBS Journal of Translational Engineering in Health and Medicine (JTEHM). Special Issue: advanced internet things personalized healthcare system validation analysis utilization. July 2018.

Intensive Care Unit Room (ICU) Activity Aspects: role, dominant/ICU objects, and dominant activity.

Log of various events and their sanitation qualifiers.

Sample ICU room Events Log


MASH: A Multiview Multimodal System for Monitoring Patient Sleep. Carlos Torres, Jeffrey C. Fried, Kenneth Rose, and B. S. Manjunath. In IEEE Transactions on Multimedia. Emerging Areas: Healthcare. May 2018.

The three views of the ICU bed. The sensor locations are selected to minimize sensor interference.

Sample Motion Summaries

Pose summarization in the mock-up ICU with 15-sec tick marks.
Pose summarization in the medical ICU with 1-min tick marks.

[PDF]  [BibTex]  [Dataset Soon]

RAM: Role Representation and Identification from Combined Appearance and Semantic Activity Maps. Carlos Torres, Archith J. Bency, Jeffrey C. Fried, and B. S. Manjunath. In Proceedings of the International Conference on Distributed Smart Cameras (ICDSC). Stanford, California. September 5-7, 2017.

Left: views of the mock-up (top) and medical (bottom) ICU; center: salient objects; right: inferred roles

Sample ICU Roles 

Sample roles demonstrating their clothing texture and color (i.e., appearance)

[PDF] [BibTex] [slides]

Deep Eye-CU (DECU): Summarization of Patient Motion in the ICU. Carlos Torres, Kenneth Rose, Jeffrey C. Fried, and B. S. Manjunath. In Proceedings of European Conference on Computer Vision (ECCV). Workshop on Assistive Computer Vision and Robotics (ACVR).  Amsterdam, The Netherlands. October 8-16, 2016.

DECU’s sleep pose temporal pattern summarization workflow.

[PDF ] [BibTex] [ArXiv] [dataset] [slides]

Eye-CU: Sleep Pose Classification for Healthcare using Multimodal Multiview Data. Carlos Torres, Victor Fragoso, Scott D. Hammond, Jeffrey C. Fried, and B. S. Manjunath. In IEEE Proceedings of the Winter Conference on Applications of Computer Vision (WACV). Lake Placid, New York. March 7-9, 2016.

Eye-CU’s multimodal multiview sensor trust estimation process and system workflow.

[PDF] [BibTex ] [ArXiv] [dataset– Bisque] [slides] WACV2016 Pocket [Guide]

MESH: Sleep Pose Recognition in an ICU From Multimodal Data and Environmental Feedback. Carlos Torres, Scott D. Hammond, Jeffrey C. Fried, and B. S. Manjunath. Computer Vision Systems: 10th International Conference (ICVS). Copenhagen, Denmark. July 6-9, 2015, Proceedings. Vol. 9163. Springer, 2015.

MESH: multimodal (RGB + Depth + Pressure) pose estimation representation and workflow.

[PDF] [BibTex] [dataset] [slides]


Silica pores and protein.

Silica as a Matrix for Encapsulating Proteins: Surface Effects on Protein Structure Assessed by Circular Dichroism Spectroscopy. Phillip J. Calabretta, Mitchell C. Chancellor, Carlos Torres, Gary R. Abel, Jr., Clayton Niehaus, Nathan J. Birtwhistle, Nada M. Khouderchah, Genet H. Zemede, and Daryl K.  Journal of functional biomaterials Vol.3. No. 3 . 514-527, 2012.

[PDF] [BibTex] — Open Access

Spectrometer plot showing ellipticity of encapsulated protein’s .

Protein Adsorption onto Organically Modified Silica Glass Leads to a Different Structure than Sol-Gel Encapsulation. B. Menaa, C. Torres, M. Herrero, V. Rives, A.R.W. Gilbert, and D.K. Eggers.  Biophysical journal. Vol.95. No. 8. L51-L53, 2008.

[PDF] [BibTex] — Open Access


  • DECU (Deep Eye-CU): Summarization of Patient Motion in the ICU. Carlos Torres, Kenneth Rose, Jeffrey C. Fried, and B. S. Manjunath. European Conference on Computer Vision (ECCV) – Workshop on Assistive Computer Vision and Robotics. Amsterdam, The Netherlands. Oct 9, 2016. [poster]
  • Eye-CU: Sleep Pose Classification for Healthcare using Multimodal Multiview Data. Carlos Torres, Victor Fragoso, Scott. D. Hammond, Jeffrey.C. Fried, and B. S. Manjunath. IEEE WACV, Lake Placid, NY. March 7 2016. [poster]
  • Embedded 3-D Systems for Action Recognition. Isaac Flores, Carlos Torres, and B. S. Manjunath. University of California Santa Barbara, Summer Undergraduate Research Colloquium, Santa Barbara, CA. Aug 15, 2013.
  • Priming Effects on Visual Scene Search by Jared Bruhn, Carlos Torres, and B. S. Manjunath. University of California Santa Barbara, INSET Undergraduate Symposium, Santa Barbara, CA. Aug 11, 2011.
  • Use of CD Spectroscopy to Assess the Biocompatibility of Silica-Based Materials. Daryl K. Eggers, Gary R. Abel, Jr., Phillip J. Calabretta, Mitchell C. Chancellor, Carlos Torres. Society for Biomaterials Annual Meeting, 2010. Seattle, WA. Apr 21–24, 2010.
  • Effects of Surface Chemistry on the Structure of Apomyoglobin in a Confined Environment. Phillip J. Calabretta, Carlos Torres, and Daryl K. Eggers.USA-Mexico Workshop in Biological Chemistry, CINVESTAV Mexico D.F., Mex. Mar 25–27, 2009.
  • Autonomous Object Placement using Force and Torque Sensors. Carlos Torres, Cressel D. Anderson, A. Jain, and Charles C. Kemp. National Institutes of Health (NIH) Annual Biomedical Research Conf. for Minority Students (ABRCMS), Orlando, FL. Nov 5–8, 2008.
  • Adsorption of Cytochrome c and Apomyoglobin to Modified Silica Glasses. Carlos Torres and Daryl K. Eggers. American Chemical Society (ACS), Undergraduate Research Symposium, Santa Clara, CA. May 03, 2008.
  • Protein Adsorption and Conformation in Hydrophobic Nanoporous Silica-based Sol-Gel Glasses. A.R.W. Gilbert, Carlos Torres, Daryl K. Eggers, and B Menaa. CSU Biotechnology Symposium, Oakland, CA. Jan 11–12, 2008.
%d bloggers like this: