Download or read book Exploring Temporal Information for Improved Video Understanding written by Yi Zhu and published by . This book was released on 2019 with total page 328 pages. Available in PDF, EPUB and Kindle. Book excerpt: In this dissertation, I present my work towards exploring temporal information for better video understanding. Specifically, I have worked on two problems: action recognition and semantic segmentation. For action recognition, I have proposed a framework, termed hidden two-stream networks, to learn an optimal motion representation that does not require the computation of optical flow. My framework alleviates several challenges faced in video classification, such as learning motion representations, real-time inference, multi-framerate handling, generalizability to unseen actions, etc. For semantic segmentation, I have introduced a general framework that uses video prediction models to synthesize new training samples. By scaling up the training dataset, my trained models are more accurate and robust than previous models even without modifications to the network architectures or objective functions. Along these lines of research, I have worked on several related problems. I performed the first investigation into depth for large-scale video action recognition where the depth cues are estimated from the videos themselves. I further improved my hidden two-stream networks for action recognition through several strategies, including a novel random temporal skipping data sampling method, an occlusion-aware motion estimation network and a global segment framework. For zero-shot action recognition, I proposed a pipeline using a large-scale training source to achieve a universal representation that can generalize to more realistic cross-dataset unseen action recognition scenarios. To learn better motion information in a video, I introduced several techniques to improve optical flow estimation, including guided learning, DenseNet upsampling and occlusion-aware estimation. I believe videos have much more potential to be mined, and temporal information is one of the most important cues for machines to perceive the visual world better.
Download or read book Video Event Recognition and Prediction Based on Temporal Structure Analysis written by Kang Li and published by . This book was released on 2015 with total page 161 pages. Available in PDF, EPUB and Kindle. Book excerpt: The increasing ubiquitousness of multimedia information in today's world has positioned video as a favored information vehicle, and given rise to an astonishing generation of social media and surveillance footage. Consumer-grade video is becoming abundant on the Internet, and it is now easier than ever to download multimedia material of any kind and quality. This raises a series of technological demands for automatic video understanding, which has motivated the research community to guide its steps towards a better attainment of such capabilities. As a result, current trends on cognitive vision promise to recognize complex events and self-adapt to different environments, while managing and integrating several types of knowledge. One important problem that will significantly enhance semantic-level video analysis is activity and event understanding, which aims at accurately describing video contents using key semantic elements, such activities and events. One well-known challenge is the long-standing semantic gap between computable low-level features and semantic information that they encode. In this thesis, several studies of high-level video content understanding were presented, which address these difficulties and narrow the semantic gap effectively. In particular, we have focused on two types of videos, namely human activity video and unconstrained consumer video. The proposed temporal structure analysis frameworks significantly extend the domains of video that can be understood by machine vision systems. In aspect of human activity recognition, we notice that in case a time-critical decision is needed, there is no work that utilizes the temporal structure of videos for early prediction of ongoing human activity. Thus we present a general activity prediction framework in which human activities can be characterized by a complex temporal composition of constituent simple actions and interacting objects. Then we extend our work to the 3D cases of action prediction motivated by recent advent of the cost-effective sensors, such as depth camera Kinect. By considering 3D action data as multivariate time series (m.t.s.) synchronized to a shared common clock (frames), we proposed a stochastic process called Marked Point Process (MPP) modelling the 3D action as temporal dynamic patterns, where both timing and strength information are captured. In aspect of unconstrained consumer video understanding, we also focus on the temporal structure of the video content through a semantic-segment based design, in which each video clip can be represented as a series of varying videography words. Then, unique videography signatures from different events can be automatically identified, using statistical analysis methods. We explore the use of videography analysis for different types of applications, including content-based video retrieval, video summarization (both visual and textual), videography based feature pooling.
Download or read book Spatio temporal Visual Information Analysis for Moving Object Detection and Retrieval in Video Sequences written by Dianting Liu and published by . This book was released on 2013 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: The development of the Internet makes the number of online videos increase dramatically, which brings new demands to the video search engines for automatic retrieval and classification. We propose an unsupervised moving object detection and retrieval framework by exploiting and analyzing spatio-temporal visual information in the video sequences. The motivation is to use visual content information to estimate the locations of the moving objects in the spatio-temporal domain. Compared with the existing approaches, our proposed detection algorithm is unsupervised. It does not need to train models for specific objects. Furthermore, it is suitable for the detection of unknown objects. Therefore, after object detection, the object-level features can be extracted for video retrieval. The proposed moving object detection algorithm consists of two layers: global motion estimation layer and local motion estimation layer. The two layers explore and estimate motion information from different scopes in the spatio-temporal domain. The global motion estimation layer uses a temporal-centered estimation method to obtain a preliminary region of motion. Specially, it analyzes the motion in the temporal domain by using our proposed novel motion representation method called the weighted histogram of Harris3D volume which combines the optical flow field and Harris3D corner detector to obtain a good spatio-temporal estimation in the video sequences. The idea is motivated by taking advantages of the two sources of motion knowledge identified by different methods to get a complementary motion data to be kept in the new motion representation. The method, considering integrated motion information, works well with the dynamic background and camera motion, and demonstrates the advantages of integrating multiple spatio-temporal cues in the proposed framework. In addition, a center-surround coherency evaluation model is proposed to compute the local motion saliency and weight the spatio-temporal motion to find the region of a moving object by the integral density algorithm. The global motion estimation layer passes the preliminary region of motion to the local motion estimation layer. The latter uses a spatial-centered estimation method to integrate visual information spatially in adjacent frames to obtain the region of the moving object. The visual information in the frame is analyzed to find visual key locations which are defined as the maxima and minima of the result of the difference-of-Gaussian function. A motion map of adjacent frames is obtained to represent the temporal information from the differences of the outcomes from the simultaneous partition and class parameter estimation (SPCPE) framework. The motion map filters visual key locations into key motion locations (KMLs) where the existence of the moving object is implied. The integral density method is employed to find the region with the highest density of KMLs as the moving object. The features extracted from the motion region are used to train the global Gaussian mixture models for the video representation. The representation significantly reduces the classification model training time in comparison to the time needed when the whole feature sets are used. It also achieves better classification performance. When combined with the information of scenes, the performance is further enhanced. Besides the proposed spatio-temporal object detection work, two other related methods are also proposed since they play subsidiary roles in the detection model. One is the innovative key frame detection method which selects representative frames as the key frames to provide key locations in the spatial-centered estimation method. By analyzing the visual differences between frames and utilizing the clustering technique, a set of key frame candidates is first selected at the shot level, and then the information within a video shot and between video shots is used to adaptively filter the candidate set to generate the final set of key frames for spatial motion analysis. Another new method is to segment and track two objects under occlusion situations, which is useful in multiple object detection scenarios.
Download or read book Spatio temporal Representation for Reasoning with Action Genome written by Kesar Murthy and published by . This book was released on 2021 with total page 38 pages. Available in PDF, EPUB and Kindle. Book excerpt: Representing Spatio-temporal information in videos has proven to be a difficult task compared to action recognition in videos involving multiple actions. A single activity consists many smaller actions that can provide better understanding of the activity. This paper tries to represent the varying information in a scene-graph format in order to answer temporal questions to obtain improved insights for the video, resulting in a directed temporal information graph. This project will use the Action Genome dataset, which is a variation of the charades dataset, to capture pairwise relationships in a graph. The model performs significantly better than the benchmark results of the dataset providing state-of-the-art results in predicate classification. The paper presents a novel Spatio-temporal scene graph for videos, represented as a directed acyclic graph that maximises the information in the scene. The results obtained in the counting task suggest some interesting finds that are described in the paper. The graph can be used for reasoning with a much lower computational requirement explored in this work among other downstream tasks such as video captioning, action recognition and more, trying to bridge the gap between videos and textual analysis.
Download or read book Action Recognition from Videos Using Deep Neural Networks written by Rishikesh Sanjay Ghewari and published by . This book was released on 2017 with total page 48 pages. Available in PDF, EPUB and Kindle. Book excerpt: Convolutional neural network(CNN) models have been extensively used in recent years to solve the problem of image understanding giving state-of-the-art results in tasks like classification, recognition, retrieval, segmentation and object detection. Motivated by this success there have been several attempts to extend convolutional neural networks for video understanding and classification. An important distinction between images and videos is the temporal information that is encoded by the sequence of frames. Most CNN models fail to capture this temporal information. Recurrent neural networks have shown promising results in modelling sequences. In this work we present a neural network model which combines convolutional neural networks and recurrent neural networks. We first evaluate the effect of the convolutional network used for understanding static frames on action recognition. Following this we explore properties that are inherent in the dataset. We combine the representation we get from the convolutional network, the temporal information we get from the sequence of video frames and other properties of the dataset to create a unified model which is trained on the UCF-101 dataset for action recognition. We evaluate our model on the pre-defined test set splits of the UCF-101 dataset. We show that our model is able to achieve an improvement over the baseline model. We show comparison between our models and various models proposed in other related works on the UCF-101 dataset. We observe that a good model for action recognition not only needs to understand static frames but also needs to encode the temporal information across a sequence of frames.
Download or read book Deep Learning for Video Understanding written by Zuxuan Wu and published by Springer Nature. This book was released on with total page 194 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Understanding Large Temporal Networks and Spatial Networks written by Vladimir Batagelj and published by John Wiley & Sons. This book was released on 2014-11-03 with total page 464 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book explores social mechanisms that drive network change and link them to computationally sound models of changing structure to detect patterns. This text identifies the social processes generating these networks and how networks have evolved. Reviews: "this book is easy to read and entertaining, and much can be learned from it. Even if you know just about everything about large-scale and temporal networks, the book is a worthwhile read; you will learn a lot about SNA literature, patents, the US Supreme Court, and European soccer." (Social Networks) "a clear and accessible textbook, balancing symbolic maths, code, and visual explanations. The authors’ enthusiasm for the subject matter makes it enjoyable to read" (JASSS)
Download or read book Multimodal Learning toward Micro Video Understanding written by Liqiang Nie and published by Springer Nature. This book was released on 2022-05-31 with total page 170 pages. Available in PDF, EPUB and Kindle. Book excerpt: Micro-videos, a new form of user-generated contents, have been spreading widely across various social platforms, such as Vine, Kuaishou, and Tik Tok. Different from traditional long videos, micro-videos are usually recorded by smart mobile devices at any place within a few seconds. Due to its brevity and low bandwidth cost, micro-videos are gaining increasing user enthusiasm. The blossoming of micro-videos opens the door to the possibility of many promising applications, ranging from network content caching to online advertising. Thus, it is highly desirable to develop an effective scheme for the high-order micro-video understanding. Micro-video understanding is, however, non-trivial due to the following challenges: (1) how to represent micro-videos that only convey one or few high-level themes or concepts; (2) how to utilize the hierarchical structure of the venue categories to guide the micro-video analysis; (3) how to alleviate the influence of low-quality caused by complex surrounding environments and the camera shake; (4) how to model the multimodal sequential data, {i.e.}, textual, acoustic, visual, and social modalities, to enhance the micro-video understanding; and (5) how to construct large-scale benchmark datasets for the analysis? These challenges have been largely unexplored to date. In this book, we focus on addressing the challenges presented above by proposing some state-of-the-art multimodal learning theories. To demonstrate the effectiveness of these models, we apply them to three practical tasks of micro-video understanding: popularity prediction, venue category estimation, and micro-video routing. Particularly, we first build three large-scale real-world micro-video datasets for these practical tasks. We then present a multimodal transductive learning framework for micro-video popularity prediction. Furthermore, we introduce several multimodal cooperative learning approaches and a multimodal transfer learning scheme for micro-video venue category estimation. Meanwhile, we develop a multimodal sequential learning approach for micro-video recommendation. Finally, we conclude the book and figure out the future research directions in multimodal learning toward micro-video understanding.
Download or read book Proceedings of the 9th Ph D retreat of the HPI Research School on service oriented systems engineering written by Meinel, Christoph and published by Universitätsverlag Potsdam. This book was released on 2017-03-23 with total page 266 pages. Available in PDF, EPUB and Kindle. Book excerpt: Design and implementation of service-oriented architectures impose numerous research questions from the fields of software engineering, system analysis and modeling, adaptability, and application integration. Service-oriented Systems Engineering represents a symbiosis of best practices in object orientation, component-based development, distributed computing, and business process management. It provides integration of business and IT concerns. Service-oriented Systems Engineering denotes a current research topic in the field of IT-Systems Engineering with high potential in academic research and industrial application. The annual Ph.D. Retreat of the Research School provides all members the opportunity to present the current state of their research and to give an outline of prospective Ph.D. projects. Due to the interdisciplinary structure of the Research School, this technical report covers a wide range of research topics. These include but are not limited to: Human Computer Interaction and Computer Vision as Service; Service-oriented Geovisualization Systems; Algorithm Engineering for Service-oriented Systems; Modeling and Verification of Self-adaptive Service-oriented Systems; Tools and Methods for Software Engineering in Service-oriented Systems; Security Engineering of Service-based IT Systems; Service-oriented Information Systems; Evolutionary Transition of Enterprise Applications to Service Orientation; Operating System Abstractions for Service-oriented Computing; and Services Specification, Composition, and Enactment.
Download or read book Medical Image Computing and Computer Assisted Intervention MICCAI 2023 written by Hayit Greenspan and published by Springer Nature. This book was released on 2023-09-30 with total page 783 pages. Available in PDF, EPUB and Kindle. Book excerpt: The ten-volume set LNCS 14220, 14221, 14222, 14223, 14224, 14225, 14226, 14227, 14228, and 14229 constitutes the refereed proceedings of the 26th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2023, which was held in Vancouver, Canada, in October 2023. The 730 revised full papers presented were carefully reviewed and selected from a total of 2250 submissions. The papers are organized in the following topical sections: Part I: Machine learning with limited supervision and machine learning – transfer learning; Part II: Machine learning – learning strategies; machine learning – explainability, bias, and uncertainty; Part III: Machine learning – explainability, bias and uncertainty; image segmentation; Part IV: Image segmentation; Part V: Computer-aided diagnosis; Part VI: Computer-aided diagnosis; computational pathology; Part VII: Clinical applications – abdomen; clinical applications – breast; clinical applications – cardiac; clinical applications – dermatology; clinical applications – fetal imaging; clinical applications – lung; clinical applications – musculoskeletal; clinical applications – oncology; clinical applications – ophthalmology; clinical applications – vascular; Part VIII: Clinical applications – neuroimaging; microscopy; Part IX: Image-guided intervention, surgical planning, and data science; Part X: Image reconstruction and image registration.
Download or read book Computer Vision ECCV 2016 Workshops written by Gang Hua and published by Springer. This book was released on 2016-11-23 with total page 938 pages. Available in PDF, EPUB and Kindle. Book excerpt: The three-volume set LNCS 9913, LNCS 9914, and LNCS 9915 comprises the refereed proceedings of the Workshops that took place in conjunction with the 14th European Conference on Computer Vision, ECCV 2016, held in Amsterdam, The Netherlands, in October 2016. The three-volume set LNCS 9913, LNCS 9914, and LNCS 9915 comprises the refereed proceedings of the Workshops that took place in conjunction with the 14th European Conference on Computer Vision, ECCV 2016, held in Amsterdam, The Netherlands, in October 2016. 27 workshops from 44 workshops proposals were selected for inclusion in the proceedings. These address the following themes: Datasets and Performance Analysis in Early Vision; Visual Analysis of Sketches; Biological and Artificial Vision; Brave New Ideas for Motion Representations; Joint ImageNet and MS COCO Visual Recognition Challenge; Geometry Meets Deep Learning; Action and Anticipation for Visual Learning; Computer Vision for Road Scene Understanding and Autonomous Driving; Challenge on Automatic Personality Analysis; BioImage Computing; Benchmarking Multi-Target Tracking: MOTChallenge; Assistive Computer Vision and Robotics; Transferring and Adapting Source Knowledge in Computer Vision; Recovering 6D Object Pose; Robust Reading; 3D Face Alignment in the Wild and Challenge; Egocentric Perception, Interaction and Computing; Local Features: State of the Art, Open Problems and Performance Evaluation; Crowd Understanding; Video Segmentation; The Visual Object Tracking Challenge Workshop; Web-scale Vision and Social Media; Computer Vision for Audio-visual Media; Computer VISion for ART Analysis; Virtual/Augmented Reality for Visual Artificial Intelligence; Joint Workshop on Storytelling with Images and Videos and Large Scale Movie Description and Understanding Challenge.
Download or read book Computer Vision ECCV 2022 written by Shai Avidan and published by Springer Nature. This book was released on 2022-10-20 with total page 815 pages. Available in PDF, EPUB and Kindle. Book excerpt: The 39-volume set, comprising the LNCS books 13661 until 13699, constitutes the refereed proceedings of the 17th European Conference on Computer Vision, ECCV 2022, held in Tel Aviv, Israel, during October 23–27, 2022. The 1645 papers presented in these proceedings were carefully reviewed and selected from a total of 5804 submissions. The papers deal with topics such as computer vision; machine learning; deep neural networks; reinforcement learning; object recognition; image classification; image processing; object detection; semantic segmentation; human pose estimation; 3d reconstruction; stereo vision; computational photography; neural networks; image coding; image reconstruction; object recognition; motion estimation.
Download or read book Computer Vision ECCV 2020 Workshops written by Adrien Bartoli and published by Springer Nature. This book was released on 2021-01-02 with total page 777 pages. Available in PDF, EPUB and Kindle. Book excerpt: The 6-volume set, comprising the LNCS books 12535 until 12540, constitutes the refereed proceedings of 28 out of the 45 workshops held at the 16th European Conference on Computer Vision, ECCV 2020. The conference was planned to take place in Glasgow, UK, during August 23-28, 2020, but changed to a virtual format due to the COVID-19 pandemic. The 249 full papers, 18 short papers, and 21 further contributions included in the workshop proceedings were carefully reviewed and selected from a total of 467 submissions. The papers deal with diverse computer vision topics. Part IV focusses on advances in image manipulation; assistive computer vision and robotics; and computer vision for UAVs.
Download or read book Advanced Intelligent Computing Technology and Applications written by De-Shuang Huang and published by Springer Nature. This book was released on with total page 533 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Pattern Recognition ICPR International Workshops and Challenges written by Alberto Del Bimbo and published by Springer Nature. This book was released on 2021-02-22 with total page 753 pages. Available in PDF, EPUB and Kindle. Book excerpt: This 8-volumes set constitutes the refereed of the 25th International Conference on Pattern Recognition Workshops, ICPR 2020, held virtually in Milan, Italy and rescheduled to January 10 - 11, 2021 due to Covid-19 pandemic. The 416 full papers presented in these 8 volumes were carefully reviewed and selected from about 700 submissions. The 46 workshops cover a wide range of areas including machine learning, pattern analysis, healthcare, human behavior, environment, surveillance, forensics and biometrics, robotics and egovision, cultural heritage and document analysis, retrieval, and women at ICPR2020.
Download or read book Temporal Data Mining written by Theophano Mitsa and published by CRC Press. This book was released on 2010-03-10 with total page 398 pages. Available in PDF, EPUB and Kindle. Book excerpt: From basic data mining concepts to state-of-the-art advances, this book covers the theory of the subject as well as its application in a variety of fields. It discusses the incorporation of temporality in databases as well as temporal data representation, similarity computation, data classification, clustering, pattern discovery, and prediction. The book also explores the use of temporal data mining in medicine and biomedical informatics, business and industrial applications, web usage mining, and spatiotemporal data mining. Along with various state-of-the-art algorithms, each chapter includes detailed references and short descriptions of relevant algorithms and techniques described in other references.
