When imaging reveals an abnormality in a kidney — such as cancer or a stone — patients and their physicians face a critical decision for what to do next: Wait? Do an invasive procedure? Or run more tests?

A new 91�Թ��� research project combines artificial intelligence (AI), computer vision and clinical expertise to help physicians make that decision with greater confidence and efficiency.

The project, called Kidney-VLM, is supported by a one-year $974,681 grant from the Florida Department of Health’s Florida Cancer Innovation Fund and is led by Pegah Khosravi, a medical AI expert and College of Medicine associate professor with a joint appointment at 91�Թ���’s College of Engineering and Computer Science.

VLM stands for vision-language model, a trainable AI platform that examines both images and data. The 91�Թ��� system analyzes visual information such as CT scans, while the language component comes from radiology, bloodwork and pathology reports. Bringing together those sources allows the system to provide a more comprehensive look at a patient’s kidney health to support more informed treatment planning. A physician could also ask the system a question and receive a response tied specifically to the patient’s imaging data and pathology.

“Kidney-VLM is designed to connect pieces of a patient’s medical story that may often be reviewed separately,” Khosravi says. “Medical AI should not merely produce an answer. It should provide evidence that allows clinicians to understand, evaluate and appropriately question that answer.”

A Clearer Vision for Patient Care

Khosravi is leading the project through , a collaborative initiative that brings together more than two dozen AI-related faculty members, their students and industry partners across disciplines. AdventHealth, a 91�Թ��� Pegasus Partner, is also partnering on the research.

Trustee Chair of Computer Science , IAI’s director and founding director of 91�Թ���’s Center for Research in Computer Vision, brings decades of experience in computer vision, visual reasoning and machine learning to the project.

He says the new research addresses the challenges of integrating computer vision and medical AI into clinical settings.

“Kidney-VLM advances computer vision by moving beyond isolated detection or classification toward an integrated and clinically interpretable system,” Shah says. “This project offers an opportunity to translate advances in computer vision into a practical tool that may help clinicians characterize kidney tumors more confidently and reduce unnecessary procedures.”

The project exemplifies 91�Թ���’s willingness to foster interdisciplinary research and continue responsibly exploring the potential of AI, he says.

“By bringing AI researchers, physicians, and clinical partners together from the beginning, the project creates a pathway from foundational research to meaningful patient impact,” Shah says. “With 91�Թ���’s strength in computer vision, the College of Medicine’s clinical expertise, and our partnership with AdventHealth, Florida is well positioned to become a national leader in responsible and clinically impactful AI innovation. This is exactly the type of collaboration we seek to foster.”

91�Թ��� computer science researchers Yu Tian and Chen Chen are helping develop the AI architecture. Tian is an assistant professor with joint appointments in the College of Engineering and Computer Science and College of Medicine, while Chen is an associate professor of computer science.

“Success will not be measured only by whether the model achieves a higher accuracy score,” Khosravi says. “We also want to determine whether it performs reliably across various patient groups and clinical settings, whether its explanations are medically sound and whether clinicians can recognize when the system is uncertain. Of course, a clinically useful AI system must be accurate, but it must also be transparent, calibrated and safe.”

Making AI Work in the Clinic

Abdo Asmar, a professor of internal medicine at the College of Medicine and a nephrologist with 91�Թ��� Health Faculty Physician Practice, is guiding the project from the clinical side. He cares for patients who may undergo imaging because of kidney disease, hypertension, kidney stones or another medical issue and then discover they have a renal mass or complex cyst.

“This project is important because it focuses on a very real clinical problem with direct implications for patients: how to make better decisions when kidney tumors or suspicious renal findings are discovered,” Asmar says. “Ultimately, this could help patients avoid unnecessary surgery, reduce healthcare costs and ensure that those with more concerning cancers receive timely evaluation and treatment. To me, this project represents a practical and patient-centered application of AI.”

He stresses that physicians must approach AI as a support, not a replacement, for medical judgment.

“We’re not replacing clinical judgment but strengthening it at the point where decisions are most consequential.” — Abdo Asmar, professor

“We’re not replacing clinical judgment but strengthening it at the point where decisions are most consequential,” he says. “While I am excited about the potential uses of AI in medicine, I remain grounded in the belief that physicians and medical researchers have an obligation to use AI carefully, ethically and always in service of better patient care.”

AdventHealth Research Institute’s Michael McDonald, a urologic oncologist, surgeon and researcher is working with the team to ensure the research remains clinically relevant and scientifically accurate.

He envisions that Kidney-VLM can address existing diagnostic limitations.

“Our ultimate goal is to provide physicians with better information before treatment decisions are made,” McDonald says. “For example, in kidney cancer, one of the major challenges is that imaging can’t always reliably determine whether a renal mass is benign or malignant before surgery, and the final diagnosis may not be known until after the tumor is removed and examined by pathology.”

Kidney-VLM is a focused tool to pinpoint a specific set of medical conditions, but Khosravi and her team are hopeful that the model will provide a foundation for broader advances in medical AI.

“The importance of this project extends beyond a single algorithm or disease,” Khosravi says. “Kidney-VLM provides an opportunity to develop a reproducible framework for trustworthy medical AI that can eventually be adapted to other cancers and clinical challenges.”

Research reported in this article was funded by the State of Florida, Department of Health, Public Health Research, Biomedical Research Program, through the Casey DeSantis Florida Cancer Innovation Fund (grant number 26C36). This content is solely the responsibility of the authors and does not necessarily represent the official views of the State of Florida.

The winning team includes Mubarak Shah, the director of the 91�Թ��� Institute of Artificial Intelligence (IAI); Helen Huang and Qiushi Fu, associate professors of biomedical engineering; Yue Wen, an assistant professor of biomedical engineering; Abhilash Durgam, a doctoral student who works in the Center for Research in Computer Vision; and Jerry Fu, a postdoctoral scholar mentored by Huang and Wen.

As top 10 winners, Team Marque’s code will be added to the competition’s open-source repository, contributing to the future advancement of EEG research. They also receive a certificate in recognition of their achievement. Shah says that placing in the top 10 at the world’s premier venue for AI and machine learning is a tremendous accomplishment for 91�Թ��� and its newly established IAI.

“It speaks to the strength of 91�Թ���’s interdisciplinary culture,” Shah says.

“Our students and faculty, with their combined expertise in machine learning, neuroscience, signal processing and computer vision can compete with some of the world’s best teams.” — Mubarak Shah, Trustee Chair Professor

The competitors had to prevail in two individual challenges that utilized data from the Healthy Brain Network, which includes EEGs of more than 3,000 children who were multitasking. Challenge 1 asked the teams to improve the predicted reaction time of a subject seeing change in contrast of an image while Challenge 2 called for an improved prediction of mental health traits in a subject.

Durgam says the secret to Team Marque’s success was to look for the patterns that hold true for all people.

“Rather than treat this as a regression problem to predict a number, we used a classification approach where we taught our model to recognize the unique ‘profile’ of the person,” Durgam says. “This encouraged the model to understand the individual’s distinct characteristics rather than just treating the task as a simple math problem.”

The team’s efforts are more than just an accomplishment for themselves and for the university — their code can now be used by scientists to advance EEG research.

“Our open-source repository supports open-science efforts, which I believe is necessary to make substantial breakthroughs in EEG research at a faster rate than any one group could accomplish alone,” Huang says. “Being able to predict mental health traits in developing children is a challenging problem that has great societal impact and could be solved faster collectively as a field by working in parallel and sharing data and code so groups don’t have to repeat something that has already been tried.”

Team Marque came together after Durgam reached out to Huang to learn more about EEG. Each of them had already formed teams for the competition, but decided to combine efforts for better results. For Huang, the competition also had a personal connection as one of the organizers, Seyed Yahya Shirazi ’21PhD, is her former student.

“I don’t think we have been in the top 10 if we didn’t combine efforts,” Huang says. “Together, we could work in parallel to explore fundamentally different approaches first to identify the most promising one and then focus on optimizing specific parameters.”

“This initiative reflects our broader commitment to grow research, scale our areas of excellence and prepare students to lead in a technology-driven future.” — Alexander N. Cartwright, 91�Թ��� President

The institute will unite more than two dozen world-class faculty across multiple colleges who are advancing foundational and applied artificial intelligence (AI) in areas like computer vision, robotics, machine learning, healthcare, finance and mathematics. By bringing AI-related faculty, students, research, and partners together, the institute will serve as a nucleus for collaboration, driving education and discovery with real-world impact.

“AI is massively transforming how we live, work and learn. As Florida’s Premier Engineering and Technology University, 91�Թ��� is uniquely positioned to lead in this critical space,” says 91�Թ��� President Alexander N. Cartwright.

With focus on research, applications-translation partnerships and education-workforce talent development, the IAI will:

• Unite an initial 25 faculty across four colleges — the College of Business, College of Engineering and Computer Science, College of Medicine, and College of Sciences — leveraging 91�Թ���’s expertise and scale to promote a truly collaborative, multidisciplinary approach.
• Promote collaboration across industry, government and community stakeholders — enhanced by 91�Թ���’s location in a thriving metropolitan region.
• Strengthen the AI skillset of all students so they can have competitive advantages as they enter the workforce.
• Support collaboration in research and education for other 91�Թ��� faculty who are doing research in AI and on its applications, opening doors learn about AI and contribute to the enhancing its applications across disciplines.

Strengthening Impact Through AI

“By bringing together leading faculty, motivated students and forward-thinking partners, the institute will unlock new possibilities across disciplines.” — Mubarak Shah, IAI director

91�Թ���’s momentum in AI is already strong. The university’s computer vision research ranks No. 8 nationally, with affiliated faculty publishing dozens of papers in premier venues and securing millions in external funding, including major awards from the Defense Advanced Research Projects Agency (DARPA), Intelligence Advanced Research Project (IARPA), U.S. National Science Foundation (NSF) and industry. In the past two years, 91�Թ���’s AI Initiative has hired 11 faculty members across multiple colleges, with eight more joining this fall. Additionally, the Center for Research in Computer Vision (CRCV) will also be joining IAI.

“The Institute of Artificial Intelligence unites 91�Թ���’s AI strengths under one roof, creating a platform for high-impact research and talent development,” says Trustee Chair of Computer Science Mubarak Shah, who has been named inaugural director of the institute.

The institute will also support 91�Թ���’s AI for All Initiative, which integrates AI into teaching and learning across disciplines. Led by Kevin Yee, special assistant to the provost for artificial intelligence, AI for All is helping to scale AI tools and curriculum across all disciplines so that students graduate with an understanding of how to effectively use AI in their careers.

About Our AI Faculty

At the core of the institute is a growing network of faculty whose excellence and expertise reflect the depth of 91�Թ���’s interdisciplinary strength in AI. Their research and teaching are defining 91�Թ���’s approach to AI as collaborative and grounded in real-world impact.

These faculty include:

• Niels da Vitoria Lobo
  Associate professor of computer science
  His research focuses on computational vision, active vision and mobile robotics, and user interfaces and graphical modeling, which has contributed to advancements in object detection, curve detection, person tracking and obstacle detection.
• Laura Brattain
  Associate professor of medicine
  Her research integrates biomedical AI, medical ultrasound and surgical robotics, with recent contributions to real-time diagnostic assistants and advancements in minimally invasive procedures. She also mentored six College of Engineering and Computer Science seniors who developed the AIMS (AI for Medical Surgery) system with Pegasus Partner Orlando Health. The system keeps track of surgical staples, enabling surgical teams to operate more efficiently and reduce waste.
• Cuneyt Akcora
  Associate professor of finance
  His research examines data science on complex networks and large-scale graph analysis, with applications in social, biological, the internet of things and blockchain networks. 91�Թ���’s future fintech involvement includes developing an AI lab within the finance department. Researchers participating in the program are looking at having blockchain nodes within the department, allowing them to download and access blockchain data, such as Bitcoin prices and activity on the transaction network. It would enable students to more easily learn about fintech and access up-to-date cryptocurrency information as they conduct research.
• Aritra Dutta
  Assistant professor of mathematics
  College of Sciences
  His research focuses on making AI models and computer systems smarter, private, faster, and more efficient. He works on improving how machines learn from large amounts of data, especially in a distributed setting, and applications in areas like image and video analysis. His interests include advanced machine learning techniques, distributed computing, and mathematical tools that help simplify learning from complex data, giving a foundational aspect to the design of advanced neural network architectures. Dutta has collaborated with major organizations such as Lockheed Martin, 1QB Information Technology, and ExxonMobil.
  
• Amrit Singh Bedi
  Assistant professor of computer science
  His research explores AI for autonomous systems. At a time when AI is rapidly reshaping industries, economies, and daily life, Bedi leads the SAFERR AI (Safety and Fairness for Reliable and Responsible AI) Lab, which aims to develop and test safer, more robust and more reliable AI systems. The interdisciplinary research team addresses critical challenges in ensuring AI systems operate dependably in real-world settings. Bedi has published over 100 papers and received numerous awards for his work on robust AI systems.

Other IAI faculty are: 

• Chen Chen
  Associate professor of computer science
  College of Engineering and Computer Science
• Xinyu Chen
  Assistant professor of mathematics
  College of Sciences
• Shahana Ibrahim
  Assistant professor of electrical and computer engineering
  College of Engineering and Computer Science
• Shubhra Kanti Karmaker
  Assistant professor of computer science
  College of Engineering and Computer Science
• Christian Küemmerle
  Assistant professor of mathematics
  College of Sciences
• Wu Lin
  Assistant professor of computer science
  College of Engineering and Computer Science
• Jialin Liu
  Assistant professor of statistics and data science
  College of Sciences
• Yogesh Rawat
  Assistant professor of computer science
  College of Engineering and Computer Science
• Yuzhang Shang
  Assistant professor of computer science
  College of Engineering and Computer Science
• Ali Siahkoohi
  Assistant professor of computer science
  College of Engineering and Computer Science
• Shashank Sonkar
  Assistant professor of computer science
  College of Engineering and Computer Science
• Yu Tian
  Assistant professor of computer science
  College of Engineering and Computer Science
• Shruti Vyas
  Assistant professor of material science and engineering
  College of Engineering and Computer Science
• Song Wang
  Assistant professor of computer science
  College of Engineering and Computer Science
• Yue Wang
  Assistant professor of electrical and computer engineering
  College of Engineering and Computer Science
• Zhenyi Wang
  Assistant professor of computer engineering
  College of Engineering and Computer Science

What Our Partners Say About the Institute

“On behalf of Leidos, I am excited to hear that 91�Թ��� is creating the Institute of AI, under the directorship of Professor Mubarak Shah. I would like to express our strong support as the AI Institute represents a significant and timely investment in national security and technological superiority. The proposed institute’s research focus — ranging the use of machine vision in multiple domains, human and AI coordination, and cyber-AI integration to develop robust security solutions is directly aligned with emerging operational challenges faced by many of Leidos’ customers.

“Leidos has collaborated with 91�Թ��� researchers on computer vision, autonomous operations, remote sensing, and other related technologies needed to further the advancement of artificial intelligence and machine learning technologies. The establishment of the IAI enables a deeper relationship between 91�Թ��� and Leidos to accelerate development and deployment of next-generation AI technologies. It would be a privilege to serve in an advisory capacity for the institute and contribute to aligning the complex challenges faced by industry with emerging AI research produced by the IAI. This ecosystem will also nurture the next wave of AI-enabled systems and train a pipeline of engineers and technologists who are essential to the future workforce.”

-Corey Hendricks, vice president and chief engineer commercial and international, Leidos

“The National Center for Simulation was founded in 1994 to promote dual-use modeling and simulation technology and is the cornerstone for the $7 billion military modeling and simulation enterprise in the Central Florida Research Park. The military’s latest objectives depend highly on AI technology, and 91�Թ��� must be utilizing leading-edge AI technology to compete for valuable research funding projects. This new technology is not only important to 91�Թ���’s Institute for Modeling and Simulation and the National Center for Simulation, but its development is critical for ensuring national security and we are thrilled to be a partner with the new 91�Թ��� Institute of AI.”

-George Cheros, president and CEO, National Center for Simulation

“At Mitsubishi Power Americas, we view AI as a transformative force — one that is essential for driving innovation, national competitiveness and societal impact. As this technology reshapes industries and redefines capabilities, the creation of this institute represents a timely and critical step toward advancing responsible, high-impact AI research and education. We look forward to the ways these advancements will revolutionize the power generation industry — through predictive maintenance, energy optimization and strengthened cybersecurity.”

“Mitsubishi Power Americas is excited to support the 91�Թ��� Institute of AI through collaborative research initiatives, helping translate innovations into real-world applications, and advancing education through internships and mentorship opportunities. By working together, we hope to contribute to a vibrant ecosystem that empowers the next generation of talent and delivers solutions to the pressing challenges faced by our organization and the global workforce.”

–Shane Mickey, senior vice president of strategic initiatives, Mitsubishi Power Americas

“91�Թ��� has been a consistent and capable partner in helping us explore advanced simulation, synthetic training environments and emerging digital capabilities. The Institute of AI directly supports our efforts to modernize and scale training through technologies like generative Al and machine learning. We appreciate the leadership role 91�Թ��� continues to play in advancing technology and talent in Central Florida, and across the nation.”

-Col. Marcus Reynolds, program manager for training systems, U.S. Marines

“The launch of the 91�Թ��� AI Institute under Dr. Shah’s leadership signals a defining moment for Central Florida and the nation. AI will shape the next era of medicine, mobility, security and society. AdventHealth is proud to stand alongside 91�Թ��� to unlock the full potential of this technology through trusted partnerships, shared innovation, and a commitment to preparing the next generation of AI leaders.”

-Rob Purinton, chief AI officer, AdventHealth

“Siemens Energy is using artificial intelligence in every aspect of our work, including expanding our Innovation Center Orlando to create a new laboratory that will use AI to analyze the electric grid. The education and workforce development that will take place at the 91�Թ��� Institute of AI will prepare students for this work and many other opportunities in our 21st century economy.”

-Rich Voorberg, president, Siemens Energy North America

“L3Harris is thrilled that 91�Թ��� is creating the Institute of AI, under the directorship of Professor Mubarak Shah. As a leading aerospace and defense technology company, L3Harris is actively advancing AI/ML (machine learning) capabilities to support national security across domains such as space systems, resilient communications and edge computing. We see strong alignment between the IAI’s research and transition goals and our interest in accelerating innovation, supporting IP (intellectual property) generation and transitioning foundational Al advances into deployable defense technologies. L3Harris looks forward to exploring future collaboration opportunities with 91�Թ��� and the new Institute of AI to help shape the next generation of AI technologies that will enhance the security, resilience and sustainability of our nation.”

-Chad Wingate, space and airborne systems AI chief scientist, L3Harris

Shah, who is the 91�Թ��� Trustee Chair Professor of Computer Science and director of the Center for Research in Computer Vision, is being recognized for his significant technical contributions and inventions. Computer vision is a science that uses complex algorithms to quickly and accurately perform computerized tasks such as facial recognition, mass-crowd counts, or detecting and identifying anomalies in images.

Shah holds eight patents in computer vision technology licensed to four companies.

Shah is 91�Թ���’s 13th NAI fellow, an honor given to those who have made a tangible impact on quality of life, economic development and the welfare of society. It is the highest professional accolade given to academic inventors. Shah holds eight patents in computer vision technology licensed to four companies.

His pioneering work in the field of automated video analysis, in collaboration with the Orlando Police Department, resulted in the KNIGHT surveillance system, which was installed in downtown Orlando. KNIGHT detects and tracks moving objects, reporting significant changes in events and activities to officers for further analysis, to give officers a new way to monitor and analyze crime-scene surveillance video footage.

Shah also developed software that calculated the world’s first automated mass-crowd count. It was first tested in 2015 by analyzing aerial photographs of protests in Catalonia, Spain, producing a count of more than 500,000 people in 30 minutes. In the future, Shah says, the technology will be used to count crowds at the Kaaba, a religious site in Saudi Arabia, during the Hajj pilgrimage to Mecca. It also may be used to help monitor crowds at the 2020 World Cup in Doha, Qatar.

Shah’s teams have conducted groundbreaking research to advance facial recognition technology. Kairos, a Miami-based company, licensed a novel algorithm by Shah and his team that performs video face recognition, identifying the same face from multiple video sources. His team also has developed a facial recognition tool that rapidly matches pictures of children with their biological parents to identify images of missing children as they age.

“This is exciting time for computer vision research in particular, and artificial intelligence in general,” says Shah. “Due to deep neural network learning disruption, we are able to solve problems today we never thought we would be able to. Computer vision is being used in medical image diagnosis, self-driving cars, manufacturing, video surveillance for safety and security, biometrics and more.”

Shah earned his doctorate from Wayne State University. In 1986, he joined 91�Թ��� where he established the computer vision program, now ranked in the top five in the United States and among the top 10 in the world. In 2006, he was honored as a Pegasus Professor, 91�Թ���’s highest faculty award.

The $999,994 grant – STEM TRansfer Students Opportunity for Nurtured Growth, or STRONG for short – was awarded to the Office of Research & Commercialization to run from next month to February 2023.

More than half of 91�Թ���’s new undergraduates are transfer students. The recipients of the new scholarships must demonstrate academic potential or ability, and demonstrate a financial need. Project STRONG will support about 30 scholarships per year in the disciplines of engineering, computer science, mathematics, physics, statistics, chemistry and biology.

“Students entering the university in STEM programs will receive the support needed to feel at home in their fields of study, at a time when they are 90 percent more likely to leave STEM than at any other time,” according to the program overview. “Talented, low-income students, of whom one out of three would otherwise leave STEM within six years, will develop interest and proficiency in STEM disciplines through opportunities provided by the program.”

91�Թ��� will provide one-on-one faculty-student mentoring, regular discipline-specific seminars, research lab internship opportunities, financial-literacy counseling and other services. The overview also says the project for STEM transfers will be conducted as “a model to be tested, improved, and ready for dissemination nationwide.”

The interdisciplinary project will be under the direction of Mubarak Shah, Trustee Chair Professor of computer science; Brian Moore, associate professor of mathematics; Malcolm B. Butler, professor of secondary education; Nazanin Rahnavar, associate professor of electric and computer engineering; and Gordon Chavis, associate vice president for enrollment services.

From this listing, 91�Թ��� ranks 21st among public universities in the nation.

The recognition is an important one because patents often lead to industrial innovations that impact daily life. Of the 56 patents 91�Թ��� obtained from the U.S. Patent and Trademark Office in 2016, more than half were related to inventions in health care – 38 percent – and optics and photonics – 18 percent – which are among 91�Թ���’s strengths and Central Florida’s regional economic priorities.

“91�Թ��� continues to compete in the top tier of the nation’s public research universities for patents because of the emphasis we place on innovation that not only elevates knowledge, but also solves real-world problems,” said Elizabeth Klonoff, 91�Թ���’s vice president for Research and dean of the College of Graduate Studies.

The report, Top 100 Worldwide Universities Granted U.S. Utility Patents in 2016, is based on the number of U.S. utility patents obtained by universities in 2016. To see the full list, visit .

Universities aim to take scientific discovery from the research bench to the marketplace, and patents are one step in that process.

91�Թ��� offers a variety of programs to help entrepreneurs reach the market. Programs include the 91�Թ��� Business Incubation Program and the Venture Accelerator, which assist with market research and business planning. I-Corps leads qualified entrepreneurial teams through a five-week “boot camp” geared toward bringing products to market, and GrowFL helps qualified companies grow to the next level.

And these programs’ success is being recognized. The Milken Institute recently ranked 91�Թ��� 22nd in the nation for its success in technology transfer.

Some examples of patents generated in 2016 include:

91�Թ���, the University of Florida and the University of South Florida, which together represent the Florida High Tech Corridor, had 261 total U.S. patents in 2016, compared to 152 granted to universities in North Carolina’s Research Triangle (Duke University, North Carolina State University and the University of North Carolina) and 234 patents granted to Silicon Hills universities in Texas (the entire University of Texas system, Rice University and Texas A&M University).

This is the fifth consecutive year that the Florida High Tech Corridor universities have outpaced other well-known research hubs nationwide.

“For an unbelievable five years in a row, The Corridor has continued to climb the ranks in terms of patents to stand alongside notable high-tech hubs across the country,” said Ed Schons, president of the Florida High Tech Corridor Council. “The outcome of this report is a testament to the work of our talented researchers, scientists and innovators, and it illustrates the promising and continued growth of our region.”

91�Թ��� has hundreds of innovations that have been developed by professors, researchers and students, ranging from diagnostics tools to displays, sensors to simulators, nanotechnology to clean tech, and many more available for licensing.

To learn more about licensing opportunities and other ways to partner with 91�Թ��� to bring inventions to market, visit .

For the first time, 91�Թ��� computer scientists will develop and test computer-vision technology that will automate the process of monitoring and reviewing thousands of hours of video streams fed from multiple cameras. The technology will be developed to work quickly to handle the large volume of data generated by the cameras, and will significantly reduce the burden placed on human investigators who perform the work and may produce faster leads for some criminal investigations.

Computer vision is a field within computer science that uses computers to quickly recognize and analyze patterns, gestures, facial features and objects in images such as photographs and videos. Cameras are already commonplace in public areas from airports to streets and the video feeds are constant.

The research team, led by Mubarak Shah, 91�Թ��� Trustee Chair professor of computer science and director of the Center for Research in Computer Vision, also includes Raymond Surette, professor of criminal justice at 91�Թ���, and researchers from Columbia University.

The team will develop the technology using archived and live video clips supplied by the Orlando Police Department to build algorithms so that computers will have the ability to recognize and flag out-of-the-norm actions, gestures, events and behaviors that could indicate criminal activity.

For example, in video footage of the 2013 Boston Marathon bombing, the suspect was the only person in the large crowd who did not look back when an explosion ignited behind him.

The project is believed to the first attempt to develop comprehensive technology that goes beyond facial recognition in collaboration with a law enforcement agency. Prior law enforcement computer vision uses have concentrated in facial recognition programs and readers, Surette said. Computer vision has not been extensively tapped and public-safety camera systems remain dependent upon a human to watch or review them.

 “Today there are too many surveillance cameras and too few human monitors,” Shah said. “Watching multiple live-video camera feeds or retroactively reviewing long hours of video streams is a mind-numbing, error-prone task.”

The technology will be applied in pre-event, live-event and post-event video timeframes, and will also employ cutting-edge search functions to speed up the amount of time it takes to review crime-scene videos for particular images.

“Utilizing the most advanced technology and tools to fight crime and keep our community safe is a top priority for the City of Orlando and this partnership will enhance these ongoing efforts,” said Orlando Police Chief John Mina. “The more eyes we have—whether they belong to officers or are created by technology—will further our mission to keep residents and visitors of Orlando safe and protected.”

The two-year project will be implemented in two phases.

During the first phase, 91�Թ��� and Columbia University researchers will develop computer vision capabilities for the surveillance work and develop user-friendly interfaces based on the needs of police investigators and data supplied by OPD. 

In the second phase, the technology will be field-tested in OPD’s camera-monitoring room with a demonstration visual-analytics workstation. The system will be linked to a new 11-camera system already active in a local neighborhood. Six cameras will be computer monitored, and five cameras will be human monitored. Surette and his team will evaluate the impact of the technology on OPD’s operations and personnel.

“With the rapid advances in computer vision in recent years, this is the perfect time to tackle this challenge,” said Shih-Fu Chang, Richard Dicker Professor and senior executive vice dean at Columbia University. “We are particularly thrilled to have the chance of combining our fast video-search solutions with the world-class activity-recognition technologies developed at 91�Թ���, in close collaboration with criminal justice experts at 91�Թ��� and the Orlando PD.” 

The actual video-surveillance technology for OPD is in development.

Under the leadership and direction of professor Mubarak Shah, a widely recognized leading expert in computer vision research, the 91�Թ��� Center for Research in Computer Vision developed the technology to significantly improve the ability to accurately complete facial identification of individuals in photos and video. The video capability is especially exciting for the industry, as traditional technologies have lacked accuracy, are typically cumbersome to operate, as well as expensive to launch and maintain.

“The capability of this facial recognition technology is very exciting to us,” said Brian Brackeen, CEO of . “Facial recognition technology has been developing over the past few decades, however, video—with its motion and variations in illumination—has long been the challenge.  Dr. Shah and his team have developed the right solution that combines accuracy, speed and deployability for the commercial market.”   ����

The technology is based on an algorithm developed by the 91�Թ��� team to focus on specific facial features that then converts a photo into a checkerboard of patches and extracted tiny snapshots of the most significant facial parts. The computer program than compares the photo against a database of photos, feature by feature, and identifies the match.  This is highly attractive within the hospitality, amusement, healthcare and electronic payment industries where organizations find it beneficial to quickly and reliably identify consumers, whether for promotional opportunities or access issues.

In its application for video, the technology provides an end-to-end face recognition system that addresses the difficult problem of identifying a video face track using a large database of still face images. Traditional technologies are based on an expensive frame-by-frame method.  Dr. Shah and his team of researchers developed a novel algorithm Mean Sequence SRC that performs video face recognition using a joint optimization leveraging all of the available video data and the knowledge that the face track frames belong to the same individual.

Whereas the defense and security industries feature a number of facial recognition technologies and companies, Kairos is targeting the enterprise market, which it believes is underserved and therefore represents a significant opportunity.  Key industries include hospitality, amusement, workforce, gaming, healthcare and electronic payment industries.

“Working with Kairos provides the perfect illustration of how 91�Թ��� research and 91�Թ���-developed technologies are not only brought to fruition but are also applied to the commercial marketplace,” said Narasimha Nagaiah (Raju), licensing associate with , who facilitated the license agreement.

Brackeen, who believes 91�Թ���’s computer vision program is one of the top programs of its kind, expressed significant satisfaction in his experience working with 91�Թ���’s Office of Technology Transfer.

“In our efforts to identify a leading-edge facial recognition technology, we were quickly able to come across

91�Թ���’s computer vision program and identify Dr. Shah’s research as groundbreaking,” said Brackeen. “The response by the Office of Technology Transfer was exceptional and thorough in its support of our effort to take this technology to the commercial market.”

In addition to working with 91�Թ���’s Office of Technology Transfer, Kairos recently received investment support through the Florida Angel Nexus and the . Kairos plans to work closely with 91�Թ��� and the computer vision program, as the company grows and adds talent.

About Kairos
Kairos is a human analytics company designed for innovative, data driven businesses. Its core business offerings provide facial recognition, crowd analytics and emotion analysis solutions. The company provides multi-dimension analysis of human beings by capturing decisive moments in people’s behavior. This intelligence empowers companies with meaningful metrics. Kairos’ mission is to radically change how companies understand people. For more information, visit,

Shah specializes in developing the theory and algorithms used for such disparate tasks as scanning crowd scenes for suspicious people, analyzing brain scans for tumors, indexing and effectively searching a large database of images and videos, and triggering a warning when a car or person approaches a railroad crossing when a train is near.

The CRCV will put 91�Թ��� in a position to capture major grants in this developing research area and support more local industries that can benefit from the technology, said MJ Soileau, vice president for research & commercialization.
“We want to build on Dr. Shah’s expertise and be in the forefront of this important field,” Soileau said.

Computer vision is the science of acquiring and processing images and videos and using computational methods for analyzing and understanding them. Since 1986 Shah has run 91�Թ���’s computer vision laboratory which has trained hundreds of students in the science behind computer vision and its use in crowd surveillance, visual tracking, human behavior analysis, determining a geo-spatial location of an image using only the contents of the image contents, unmanned aerial video analysis and bio-medical image analysis.

Shah relies extensively on algorithms which allow computers to duplicate the analytical abilities of humans, only much faster and without error.

Shah has worked with a wide range of industries including aerospace, biomedical, and modeling and simulation.

The work is important to many high-tech industries because it allows scientists to rapidly monitor and leverage large amounts of video without constant human monitoring.

“The proliferation of video sensors creates more visual data than humans can effectively review, categorize, and understand. The algorithms being developed by Dr. Shah’s team help automate the process, providing ‘intelligent triage’ to focus on important events and analyze information over long time intervals” said Jeff Pridmore, vice president of Applied Research at Lockheed Martin Missiles and Fire Control.

Shah said his vision for the CRCV is to hire world class faculty – three positions have initially been approved – and encourage existing faculty and industry leaders to utilize the center.

“The more resources we have the more high-quality students and faculty we can attract and this will help us move to the next era of computer vision research and education at 91�Թ���. This will also increase our capacity for large projects in order to provide world-class research to local high-tech industries” he said.

Specifically Shah is interested in working with faculty at 91�Թ���’s College of Medicine on projects such as automatic tracking of E. coli bacteria and early cancer detection, and with the Institute for Simulation and Training and fellow researchers in the College of Engineering and Computer Science on developing robots and driverless cars navigated by computerized sensors.

“There are many areas we can use the technology once we have the bandwidth,” he said.

In addition to starting the computer vision lab, Shah has run 91�Թ���’s longest Research Experience for Undergraduates program funded by the National Science Foundation, bringing dozens of high school students to 91�Թ��� each summer and immersing them in the research environment.

He has been honored with several awards, including Pegasus Professor, the highest award given by 91�Թ��� to a faculty member who has made a significant impact on the university. He has a Ph.D. from Wayne State University and is a fellow of IEEE, AAAS, IAPR and SPIE, the largest professional organizations in his field.

University Award for Excellence in Undergraduate Teaching: Christopher L. Parkinson, College of Sciences. Parkinson is an associate professor of biology, who is passionate about teaching. He’s been at 91�Թ��� since 2001. Even though he often teaches the same courses, he prepares each lecture with time and care. An expert in snakes and turtles, and a recipient of grants from the U.S. Fish and Wildlife Service and Florida Fish and Wildlife Conservation Commission among others, Parkinson keeps his work current and challenging for students. He has mentored or directly worked on science research with more than 50 students from high school to graduate school. And he works tirelessly to help encourage all students – even those in high school – to pursue careers in Science, Technology, Engineering and Mathematics (STEM).  Aside from active research, teaching and helping students one-on-one, Parkinson also serves on multiple university committees and provides seminars around the world.

Winners from each college were: Patricia B. Angley, Robert Cassanello, Keith Folse and Humberto López Cruz, College of Arts and Humanities; Honghui Chen and Lauryn A. Migenes, College of Business Administration; Gina Gresham and Lee-Anne Trimble Spalding, College of Education; Arup Guha and Seetha Raghavan, College of Engineering and Computer Science; Denise Gammonley and Ross Wolf, College of Health and Public Affairs; Jill Fjelstul, Rosen College of Hospitality Management; Robert A. Borgon, College of Medicine; Vicki Loerzel, College of Nursing; and Costas J. Efthimiou, Terri Susan Fine, Houman A. Sadri and John J. Schultz, College of Sciences.

University Award for Excellence in Graduate Teaching: Julia Listengarten, College of Arts and Humanities. Listengarten is an accomplished professor and sought after for her expertise in theatre by schools and centers around the globe. She integrates theory and practice in her teaching philosophy, which results in a high-intensity environment. This in turn leads to impressive student success. Some of students have gone on to work at the Kennedy Center in Washington, D.C., while others are leading children’s programs for large cities across the nation. She also shares what she knows by serving in the college’s Graduate Standards and Research Committee as well as a host of other university-wide projects.

Winners from each college were: Charles Kelliher, College of Business Administration; Haiyan Bai, College of Education; Steven J. Duranceau, College of Engineering and Computer Science; Reid M. Oetjen, College of Health and Public Affairs; Kevin Murphy, Rosen College of Hospitality Management; Steven N. Ebert, College of Medicine; Diane Wink, College of Nursing; Peter J. Delfyett Jr., College of Optics and Photonics; and Liz Grauerholz, College of Sciences.

University Award for Excellence in Research: Mubarak Shah, College of Engineering and Computer Science. Shah is a computer science professor, holds the Agere Chair in Computer Science, and is a leader in his field. During the past  five years, Shah has made fundamental contributions in two main areas of computer vision: video surveillance and visual crowd analysis. His group has developed a series of novel methods for detection of humans and vehicles, tracking of those objects and recognition of their behavior in wide-area surveillance videos obtained by Unmanned Aerial Vehicles. Shah also has started a new area of research — visual analysis of large crowds. His group has developed first-of-its-kind algorithms for segmentation of large crowds using Lagrangian dynamics and tracking of an individual in a high-density crowd. In 2010, he was selected as a fellow in the American Association for the Advancement of Science (AAAS).

Winners from each college were: Tison Pugh, College of Arts and Humanities; Michael R. Caputo, College of Business Administration; Glenn W. Lambie, College of Education; Thomas A. Bryer, College of Health and Public Affairs; H.G. Parsa, Rosen College of Hospitality Management; Dinender K. Singla, College of Medicine; Aristide Dogariu, College of Optics and Photonics; Joseph Harrington, College of Sciences; and Nazim Muradov, Institutes and Centers.

University Award for Excellence in Undergraduate Faculty Advising: Alice Noblin, College of Health and Public Affairs. Noblin is the program director for the Health Informatics and Information Management Program at the university. She advises and instructs students in this program and helps them get ready for national testing. She is excellent at getting them ready. During the most recent nine-month reporting period, the pass rate on the national certification exam for 91�Թ��� graduates was 100 percent while the national average pass rate was 74percent.  The 91�Թ��� alum also serves on many advisory committees including Health Informatics and Information Management (HIIM) Program Advisory Committee, RHIT Program Advisory Committee, and the Regional Extension Center Advisory Council. She’s a leader in the field and is respected by the students she serves.

University Award for Excellence in Professional Academic Advising: Dena M. Ford, College of Sciences. Ford is the coordinator of undergraduate academic support services. In her role she advises hundreds of undergraduate students. Having been a first-generation student and transfer student herself, she can relate to those going through the process. That’s one reason she created retention outreach workshops to help ease students’ transition to college academics. She also is responsible for all transfer students and runs their orientation sessions. She has a master’s degree in mental-health counseling and another degree in psychology, which helps her steer students in the right direction based on individual career goals. And she’s conducted research into transfer shock and different populations in higher education.

University Award for Excellence in Librarianship: Timothy Bottorff, Rosen College of Hospitality Management. Bottorff has been with the Universal Orlando Foundation Library at Rosen since its inception in 2004, first as reference librarian and then as head librarian from mid-2007 to the present. Bottorff oversees all functions of the Rosen College library, including administration, reference, library instruction, collection development, website design, marketing, circulation and other services. He is also active in the Florida Library Association and in a variety of other service, research and professional development activities within the library field.

University Awards for Excellence in Professional Service: Keith Koons, College of Arts and Humanities, and Linda Walters, College of Sciences.

Koons is a professor in the music department. His service to the university and the world of music has been outstanding. Koons was the driving force behind the creation of the music department’s Master of Arts degree and he continues to guide the program as graduate coordinator. He has served as chair of several search committees and is a long time chair of the department’s promotion and tenure committee. He has served in the Faculty Senate for years helping to shape many of the policies that have improved conditions on the campus and in the curriculum. He’s done all this while actively contributing to the International Clarinet Association. Recognizing his abilities and dedication, the association named Koons president in 2008 for a term that concludes in 2012. He also serves on many other local and national organizations related to music including the Advisory Board for A Gift for Music. The nonprofit organization provides stringed-instrument instruction to more than 450 Orange County third-, fourth- and fifth-graders who normally could not afford to be involved in a stringed-instrument program.

Walters is a professor of biology who has dedicated her career to teaching, research and service. She’s been awarded more than 50 awards specifically tied to service to the community with an estimated value of $3.1 million. Her community-based oyster reef restoration project in Volusia and Brevard counties has led to 50 reefs being restored in four years with the help of 21,936 volunteers that she organized. She’s also planned and executed 27 workshops or training sessions for educators in marine conservation in the past five years. That’s in addition to a whale watch program she ran, which benefited 615 91�Թ��� students. She’s worked with more than six local groups to provide information and education about marine biology. In addition to mentoring new faculty, she provides her expertise to the Faculty Center for Teaching and Learning, where she is a fellow. This year she also is serving as the interim director of the 91�Թ��� Women’s Research Center in addition to her teaching duties and her continuing research and oyster restoration work.

Recognition of Service to the University, 40 Years of Service: Richard C. Crepeau and Donald E. Jones, College of Arts & Humanities; Tom Mullin, College of Health and Public Affairs; John (Jeff) Butler and Roger B. Handberg and Jack M. McGuire, College of Sciences.

Recognition of Service to the University, 30 Years of Service: Lee Anne Kirkpatrick, Philip H. Pollock III, James R. Schott and Susan C. Schott, College of Sciences; Sheri Dressler, Office of Undergraduate Studies; David Beal and Betsy L. Gray and John L. Harrison, Division of Research and Commercialization.

Recognition of Service to the University, 20 Years of Service: Donald W. Seay, College of Arts & Humanities; Jeffrey S. Kaplan and Barbara A. Murray, College of Education; Haitham M. Al-Deek, Ruey-Hung Chen, Niels da Vitoria Lobo, Ronald F. DeMara and Pamela McCauley Bush, College of Engineering and Computer Science; Carol M. Bast, College of Health and Public Affairs; Weili Luo, College of Sciences; and Thomas O’Neal and Lawrence E. Ziock, Division of Research and Commercialization.

Retired and Retiring Faculty :

Jagdish J. Chavda and Ronnie Z. Hawkins, College of Arts & Humanities; Robert C. Ford, Warren McHone, Linda Beats Putchinski and William Weaver, College of Business Administration; Lawrence Mione, Jennifer Platt, Mary K. Romjue and Mary Starzynski, College of Education; Lesia L. Crumpton-Young, College of Engineering and Computer Science; Patricia Bishop, College of Graduate Studies; David Fabianic and Steve Shideler, College of Health and Public Affairs; Susan Scott Ricci, College of Nursing; Glenn D. Boreman and James E. Harvey, College of Optics and Photonics; Carol C. Bledsoe, Constance E. Cutchins, Joe Hall, Ronnie F. Smith, Brian P. Tonner and Carrie Whitcomb, College of Sciences; Thomas L. Clarke and James C. Huggins, Division of Research and Commercialization; Evelyn S. Case and Carole Hinshaw, Information Technologies and Resources; and Mary D. Bozeman and Helen Y. Hill, Regional Campuses.