Abstract: In nowadays state of the art VR environments, displayed in CAVEs or HMDs, navigation technics may frequently induce cybersickness or VR-Induced Symptoms and Effects (VRISE), drastically limiting the friendly use of VR environments with no navigation limitations. In two distinct experiments, we investigated acceleration VRISE thresholds for longitudinal and rotational motions and compared 3 different VR systems: 2 CAVEs and a HMD (Oculus Rift DK2). We found that VRISE occur more often and more strongly in case of rotational motions and found no major difference between the CAVEs and the HMD. Based on the obtained thresholds we developed a new "Head Lock" navigation method for rotational motions in a virtual environment in order to generate a “Pseudo AR” mode, keeping fixed visual outside world references. Thanks to a third experiment we have shown that this new metaphor significantly reduces VRISE occurrences and may be a useful base for future natural navigation technics.

Abstract: Feature-based steganalysis has been an integral tool for detecting the presence of steganography in communication channels for a long time. In this paper, we explore the possibility to utilize powerful optimization algorithms available in convolutional neural network packages to optimize the design of rich features. To this end, we implemented a new layer that simulates the formation of histograms from truncated and quantized noise residuals computed by convolution. Our goal is to show the potential to compactify and further optimize existing features, such as the projection spatial rich model (PSRM). Motivation In steganography, the main goal is to communicate secretly through an overt channel. Steganalysis, on the other hand tries to detect the presence of steganography. For a long time, detection relied on classifiers trained in a supervised fashion on examples of cover and stego images [3]. To detect content-adaptive schemes, researchers hand crafted various high-dimensional feature representations (rich models). Such features are typically formed from noise residuals extracted from the input image by convolutions with high-pass linear filters (kernels). The array of residuals would then be represented either with co-occurrence matrices (empirical joint densities) [19, 12, 2] or via histograms of projections on random directions [6]. Due to the inherent complexity of digital images, the design of suitable kernels has been based entirely on heuristics. While there were attempts to optimize the kernels by parametrizing them and determining the parameters by minimizing the classifier detection error using, e.g., the Nelder–Mead algorithm [5], the complexity of evaluating the objective function (training a classifier on thousands of images and evaluating its performance using, e.g., the minimal total error probability PE) makes such approaches non-scalable and unable to optimize a sufficient number of kernels to build a rich model. The main flaw is the need for a constant feedback from the classifier, which is very time consuming due to the complexity of training with highdimensional features on thousands of images. The main advantage of Convolutional Neural Networks (CNNs) is their ability to optimize the feature extraction and classification steps simultaneously and thus close the loop between feature extraction and classification. Such networks are not only capable of learning the best decision boundary between different classes but also the best representation of each data class that would improve their separability. Originally developed for numerous computer vision problems, these networks were recently adapted for steganalysis [16, 14, 13, 17, 18]. Since in a typical computer vision problem, CNNs are used to learn patterns or objects, in steganalysis the signal of interest is hidden within the noise component of the image. In order to deal with this issue, in one of the early works in this direction Qian et al. [14] proposed to adjust the network design by using one of the successfully hand-designed high-pass kernels in the Spatial Rich Model (SRM) [3] as the first (fixed) convolutional layer in the network. The main reason behind this step is to suppress most of each image content and thus force the network to “pay attention” to high-frequency details. A closer look at the CNN structure reveals a similarity with feature-based steganalysis. The convolutional layers play the role of residual extraction while activation and pooling layers mimick truncation and quantization. Various researches tried to improve the performance of CNNs for steganalysis using insights acquired from feature-based steganalysis. In one of the most recent works in this direction, Xu et al. [17] proposed a novel CNN architecture capable of approaching the performance of feature-based steganalysis. In their five-layer CNN, after initial high-pass filtering and the first convolutional layer, they proposed to take the absolute value (ABS) of the feature maps (residuals) as the activation function followed by a hyperbolic tangent (TanH) in order to preserve the sign symmetry of the residuals in a process similar to computing SRM residuals. Additionally, by using 1×1 convolutional kernels in the last three layers, the authors forced the network to collect the local statistics from the feature maps in a pixel by pixel fashion. Inspired by similarities between conventional featurebased and CNN detectors, in this paper we investigate the possibility to use existing infrastructure behind CNNs to optimize the design of kernels (linear pixel predictors) in feature-based steganalysis. The replacement of the objective function in the form of some scalar classifier performance criterion by the loss function in a CNN, powerful gradient descend algorithms could be used for the optimization task provided there was a way to form histograms within a CNN. Thus, as our first step, we implemented a histogram layer for the Caffe CNN package. We use meanshifted Gaussian functions as the building blocks of this layer to obtain a proper back-flow of gradients through the layer and to facilitate learning of the parameters behind this layer. As a proof of concept, in this paper we use this layer to model individual submodels of the PSRM [6]. Instead of forming co-occurrences of neighboring quantized residual samples, PSRM projects unquantized residual values on random directions, which are subsequently quantized and represented using histograms as steganalysis features. In PSRM, each SRM kernel is projected on 55 random two-dimensional kernels and their rotated and mirrored versions. The higher detection rate of this feature set is the result of a large number of projections, which comes at a high computational cost. This problem can render this powerful feature set unusable for applications with limited time and computational power [10]. Modeling these submodels within the CNN framework with the histogram layer enables us to reduce the high dimensionality of this feature set by replacing random kernels with fewer optimized kernels. Our study also hints at the possibility to extract more information in the final layers of CNNs to pave the way for better network designs. In the next section, we introduce the histogram layer and discuss its design and internal building blocks. In Section “Experimental setup,” we simulate PSRM models within the CNN framework using the histogram layer and outline the training procedure. The proposed histogram layer is tested in Section “Analyzing results,” where we compare the detection results of our CNN model with PSRM submodels and discuss the results. The paper is summarized in the last section. Histogram layer To capture the local statistics of feature maps using histograms without any information loss we need to use step functions centered on each histogram bin. While shifted step functions are the best choice to compute independent histogram bins, they are unusable within a CNN framework because their derivative is zero everywhere except for the edges. This prevents the back flow of gradients through the layer and the back-propagation algorithm stops working. In contrast, a Gaussian kernel seems to be a good candidate to form histogram bins. Unlike the sharp edges of a step function, its smooth slopes will create a fine path for gradients to flow backwards through each histogram bin towards previous layers. A Gaussian activation function has the form g(x) = e− (x−μ)2 σ2 (1) where μ is the center of the histogram bin and σ controls the tradeoff between the accuracy of each binning operation (and the overlap between adjacent bins) and the flow of gradients through the layer. In our experiments, we fixed σ = 0.6 to obtain a close match between the exact histogram and a histogram computed using a Gaussian activation function. Fig. 1 shows the structure of an 8-bin histogram function using Gaussian activations. The value of μ for each kernel is chosen from the set μ∈ {−3.5,−2.5, . . . ,2.5,3.5}. The tails of the Gaussian kernels on the sides are replaced with a constant value of 1 in order to simulate the residual truncation done in featurebased steganalysis. To compute the value of the histogram bin B(k) centered at μ= μk for an M ×N activation map −3.5 −2.5 −1.5 −0.5 0.5 1.5 2.5 3.5 0 0.2 0.4 0.6 0.8 1 Figure 1. Structure of an 8-bin histogram using mean-shifted Gaussian

Abstract: Current virtual environments rely heavily on audio and visual feedback as a form of sensory feedback [1]. The degree of immersion can be increased by augmenting synthetic haptic feedback from the user interface. Most of the existing wearable haptic feedback systems use tactile stimulation by vibrating motors for haptic feedback which lack a compelling sense of immersion with force feedback[2][3].e.g. in the case of pressing a button. This research addresses this issue with hardware architecture for kinesthetic force feedback. This research focuses on the design of a wearable soft robotic haptic feedback glove for force feedback in virtual environments. The glove provides a force feedback to the fingers while clicking a button in virtual environments. The glove design includes a soft exoskeleton actuated by Mckibben muscles which are controlled using a custom fluidic control board [4]. The user’s fingers are tracked using the infrared cameras. This tracking system provides the information for the position of the user’s fingers. Based on this information, the soft glove is actuated to provide a haptic feedback. The Soft exoskeleton and actuation make the glove compliant, compact and unintimidating as compared to force feedback glove with rigid kinematic linkages. The glove design is inexpensive, mass-manufacturable and compatible to 90% of the U.S. population. The user could test the glove by playing the piano in virtual reality environment. The presence of audio, visual and haptic feedback makes the virtual reality environment highly immersive. The informal pilot study indicates that haptic glove improves the immersive experience of the virtual reality environments. Users in informal pilot study described the experience as “like nothing seen before”, “mesmerizing” and “amazing”.

Abstract: It is widely recognized that steganography with sideinformation in the form of a precover at the sender enjoys significantly higher empirical security than other embedding schemes. Despite the success of side-informed steganography, current designs are purely heuristic and little has been done to develop the embedding rule from first principles. Building upon the recently proposed MiPOD steganography, in this paper we impose multivariate Gaussian model on acquisition noise and estimate its parameters from the available precover. The embedding is then designed to minimize the KL divergence between cover and stego distributions. In contrast to existing heuristic algorithms that modulate the embedding costs by 1–2|e|, where e is the rounding error, in our model-based approach the sender should modulate the steganographic Fisher information, which is a loose equivalent of embedding costs, by (1–2|e|)^2. Experiments with uncompressed and JPEG images show promise of this theoretically well-founded approach. Introduction Steganography is a privacy tool in which messages are embedded in inconspicuous cover objects to hide the very presence of the communicated secret. Digital media, such as images, video, and audio are particularly suitable cover sources because of their ubiquity and the fact that they contain random components, the acquisition noise. On the other hand, digital media files are extremely complex objects that are notoriously hard to describe with sufficiently accurate and estimable statistical models. This is the main reason for why current steganography in such empirical sources [3] lacks perfect security and heavily relies on heuristics, such as embedding “costs” and intuitive modulation factors. Similarly, practical steganalysis resorts to increasingly more complex high-dimensional descriptors (rich models) and advanced machine learning paradigms, including ensemble classifiers and deep learning. Often, a digital media object is subjected to processing and/or format conversion prior to embedding the secret. The last step in the processing pipeline is typically quantization. In side-informed steganography with precover [21], the sender makes use of the unquantized cover values during embedding to hide data in a more secure manner. The first embedding scheme of this type described in the literature is the embedding-while-dithering [14] in which the secret message was embedded by perturbing the process of color quantization and dithering when converting a true-color image to a palette format. Perturbed quantization [15] started another direction in which rounding errors of DCT coefficients during JPEG compression were used to modify the embedding algorithm. This method has been advanced through a series of papers [23, 24, 29, 20], culminating with approaches based on advanced coding techniques with a high level of empirical security [19, 18, 6]. Side-information can have many other forms. Instead of one precover, the sender may have access to the acquisition oracle (a camera) and take multiple images of the same scene. These multiple exposures can be used to estimate the acquisition noise and also incorporated during embedding. This research direction has been developed to a lesser degree compared to steganography with precover most likely due to the difficulty of acquiring the required imagery and modeling the differences between acquisitions. In a series of papers [10, 12, 11], Franz et al. proposed a method in which multiple scans of the same printed image on a flat-bed scanner were used to estimate the model of the acquisition noise at every pixel. This requires acquiring a potentially large number of scans, which makes this approach rather labor intensive. Moreover, differences in the movement of the scanner head between individual scans lead to slight spatial misalignment that complicates using this type of side-information properly. Recently, the authors of [7] showed how multiple JPEG images of the same scene can be used to infer the preferred direction of embedding changes. By working with quantized DCT coefficients instead of pixels, the embedding is less sensitive to small differences between multiple acquisitions. Despite the success of side-informed schemes, there appears to be an alarming lack of theoretical analysis that would either justify the heuristics or suggest a well-founded (and hopefully more powerful) approach. In [13], the author has shown that the precover compensates for the lack of the cover model. In particular, for a Gaussian model of acquisition noise, precover-informed rounding is more secure than embedding designed to preserve the cover model estimated from the precover image assuming the cover is “sufficiently non-stationary.” Another direction worth mentioning in this context is the bottom-up model-based approach recently proposed by Bas [2]. The author showed that a high-capacity steganographic scheme with a rather low empirical detectability can be constructed when the process of digitally developing a RAW sensor capture is sufficiently simplified. The impact of embedding is masked as an increased level of photonic noise, e.g., due to a higher ISO setting. It will likely be rather difficult, however, to extend this approach to realistic processing pipelines. Inspired by the success of the multivariate Gaussian model in steganography for digital images [25, 17, 26], in this paper we adopt the same model for the precover and then derive the embedding rule to minimize the KL divergence between cover and stego distributions. The sideinformation is used to estimate the parameters of the acquisition noise and the noise-free scene. In the next section, we review current state of the art in heuristic side-informed steganography with precover. In the following section, we introduce a formal model of image acquisition. In Section “Side-informed steganography with MVG acquisition noise”, we describe the proposed model-based embedding method, which is related to heuristic approaches in Section “Connection to heuristic schemes.” The main bulk of results from experiments on images represented in the spatial and JPEG domain appear in Section “Experiments.” In the subsequent section, we investigate whether the public part of the selection channel, the content adaptivity, can be incorporated in selection-channel-aware variants of steganalysis features to improve detection of side-informed schemes. The paper is then closed with Conclusions. The following notation is adopted for technical arguments. Matrices and vectors will be typeset in boldface, while capital letters are reserved for random variables with the corresponding lower case symbols used for their realizations. In this paper, we only work with grayscale cover images. Precover values will be denoted with xij ∈ R, while cover and stego values will be integer arrays cij and sij , 1 ≤ i ≤ n1, 1 ≤ j ≤ n2, respectively. The symbols [x], dxe, and bxc are used for rounding and rounding up and down the value of x. By N (μ,σ2), we understand Gaussian distribution with mean μ and variance σ2. The complementary cumulative distribution function of a standard normal variable (the tail probability) will be denoted Q(x) = ∫∞ x (2π)−1/2 exp ( −z2/2 ) dz. Finally, we say that f(x)≈ g(x) when limx→∞ f(x)/g(x) = 1. Prior art in side-informed steganography with precover All modern steganographic schemes, including those that use side-information, are implemented within the paradigm of distortion minimization. First, each cover element cij is assigned a “cost” ρij that measures the impact on detectability should that element be modified during embedding. The payload is then embedded while minimizing the sum of costs of all changed cover elements, ∑ cij 6=sij ρij . A steganographic scheme that embeds with the minimal expected cost changes each cover element with probability βij = exp(−λρij) 1 +exp(−λρij) , (1) if the embedding operation is constrained to be binary, and βij = exp(−λρij) 1 +2exp(−λρij) , (2) for a ternary scheme with equal costs of changing cij to cij ± 1. Syndrome-trellis codes [8] can be used to build practical embedding schemes that operate near the rate–distortion bound. For steganography designed to minimize costs (embedding distortion), a popular heuristic to incorporate a precover value xij during embedding is to modulate the costs based on the rounding error eij = cij − xij , −1/2≤ eij ≤ 1/2 [23, 29, 20, 18, 19, 6, 24]. A binary embedding scheme modulates the cost of changing cij = [xij ] to [xij ] + sign(eij) by 1−2|eij |, while prohibiting the change to [xij ]− sign(eij): ρij(sign(eij)) = (1−2|eij |)ρij (3) ρij(−sign(eij)) = Ω, (4) where ρij(u) is the cost of modifying the cover value by u∈ {−1,1}, ρij are costs of some additive embedding scheme, and Ω is a large constant. This modulation can be justified heuristically because when |eij | ≈ 1/2, a small perturbation of xij could cause cij to be rounded to the other side. Such coefficients are thus assigned a proportionally smaller cost because 1− 2|eij | ≈ 0. On the other hand, the costs are unchanged when eij ≈ 0, as it takes a larger perturbation of the precover to change the rounded value. A ternary version of this embedding strategy [6] allows modifications both ways with costs: ρij(sign(eij)) = (1−2|eij |)ρij (5) ρij(−sign(eij)) = ρij . (6) Some embedding schemes do not use costs and, instead, minimize statistical detectability. In MiPOD [25], the embedding probabilities βij are derived from their impact on the cover multivariate Gaussian model by solving the following equation for each pixel ij: βijIij = λ ln 1−2βij βij , (7) where Iij = 2/σ̂4 ij is the Fisher information with σ̂ 2 ij an estimated variance of the acquisition noise at pixel ij, and λ is a Lagrange multiplier determined by the payload size. To incorporate the side-information, the sender first converts the embedding probabilities into costs and then modulates them as in (3) or (5). This can be done b

Abstract: Understanding complex events from unstructured video, like scoring a goal in a football game, is an extremely challenging task due to the dynamics, complexity and variation of video sequences. In this work, we attack this problem exploiting the capabilities of the recently developed framework of deep learning. We consider independently encoding spatial and temporal information via convolutional neural networks and fusion of features via regularized Autoencoders. To demonstrate the capacities of the proposed scheme, a new dataset is compiled, composed of goal and no-goal sequences. Experimental results demonstrate that extremely high classification accuracy can be achieved, from a dramatically limited number of examples, by leveraging pretrained models with fine-tuned fusion of spatio-temporal features. Introduction Analyzing unstructured video streams is a challenging task for multiple reasons [10]. A first challenge is associated with the complexity of real world dynamics that are manifested in such video streams, including changes in viewpoint, illumination and quality. In addition, while annotated image datasets are prevalent, a smaller number of labeled datasets are available for video analytics. Last, the analysis of massive, high dimensional video streams is extremely demanding, requiring significantly higher computational resources compared to still imagery [11]. In this work, we focus on the analysis of a particular type of videos showing multi-person sport activities and more specifically football (soccer) games. Sport videos in general are acquired from different vantage points and the decision of selecting a single stream for broadcasting is taken by the director. As a result, the broadcasted video stream is characterized by varying acquisition conditions like zooming-in near the goalpost during a goal and zooming-out to cover the full field. In this complex situation, we consider the high level objective of detecting specific and semantically meaningful events like an opponent team scoring a goal. Succeeding in this task will allow the automatic transcription of games, video summarization and automatic statistical analysis. Despite the many challenges associated with video analytics, the human brain is able to extract meaning and provide contextual information in a limited amount of time and from a limited set of training examples. From a computational perspective, the process of event detection in a video sequence amounts to two foundamental steps, namely (i) spatio-temporal feature extraction and (ii) example classification. Typically, feature extraction approaches rely on highly engineered handcrafted features like the SIFT, which however are not able to generalize to more challenging cases. To achieve this objective, we consider the state-of-theart framework of deep learning [18] and more specifically the case of Convolutional Neural Networks (CNNs) [16], which has taken by storm almost all problems related to computer vision, ranging from image classification [15, 16], to object detection [17], and multi-modal learning [6]. At the same time, the concept of Autoencoders, a type of neural network which tries to appropriate the input at the output via regularization with various constrains, is also attracting attention due to its learning capacity in cases of unsupervised learning [21]. While significant effort has been applied in designing and evaluating deep learning architectures for image analysis, leading to highly optimized architectures, the problem of video analysis is at the forefront of research, where multiple avenues are explored. The urgent need for video analytics is driven by both the wealth of unstructured videos available online, as well as the complexities associated with adding the temporal dimension. In this work, we consider the problem of goal detection in broadcasted low quality football videos. The problem is formulated as a binary classification of short video sequences which are encoded though a spatiotemporal deep feature learning network. The key novelties of this work are to: • Develop a novel dataset for event detection in sports video and more specifically, for goal detection is football games; • Investigate deep learning architectures, such as CNN and Autoencoders, for achieving efficient event detection; • Demonstrate that learning, and thus accurate event detection, can be achieved by leveraging information from a few labeled examples, exploiting pre-trained models. State-of-the-art For video analytics, two major lines of research have been proposed, namely frame-based and motion-based, where in the former case, features are extracted from individual frames, while in the latter case, additional information regarding the inter-frame motion, like optical flow [3], is also introduced. In terms of single frame spatial feature extraction, CNNs have had a profound impact in image recognition, scene classification, and object detection, among others [16]. To account for the dynamic nature of video, a recently proposed concept involves extenting the two-dimensional convolution to three dimensions, leading to 3D CNNs, where temporal information is included as a distinct input [12, 13]. An alternative approach for encoding the temporal informarion is through the use of Long-Short Term Memory (LSTM) networks [1, 13], while another concept involves the generation of dynamic images through the collapse of multiple video frames and the use of 2D deep feature exaction on such representations [7]. In [2], temporal information is encoded through average pooling of frame-based descriptors and Figure 1: Block diagram of the proposed Goal detection framework. A 20-frame moving window initially selects part of the sequence of interest, and the selected frames undergo motion estimation. Raw pixel values and optical flows are first independently encoded using the pre-trained deep CNN for extracting spatial and temporal features. The extracted features can either be introduced into a higher level network for fusion which is fine-tuned for the classification problem, or concatenated and used as extended input features for the classification. the subsequent encoding in Fisher and VLAD vectors. In [4], the authors investigated deep video representation for action recognition, where temporal information was introduced in the frame-diff layer of the deep network architecture, through different temporal pooling strategies applied in patch-level, frame-level, and temporal window-level. One of the most successful frameworks for encoding both spatial and temploral information is the two-stream CNN [8]. Two-stream networks consider two sources of information, raw frames and optical flow, which are independently encoded by a CNN and fused into an SVM classifier. Further studies on this framework demonstrated that using pre-trained models can have a dramatic impact on training time, for the spatial and temporal features [22], while convolutional two-stream network fusion was recently applied in video action recognition [23]. The combination of 3D convolutions and the two-stream approach was also recently reported for video classification, achieving state-of-theart performance at significantly lower processing times [24]. The performance demonstrated by the two-streams approach for video analysis led to the choice of this paradigm in this work. Event Detection Network The proposed temporal event detection network is modeled as a two-stream deep network, coupled with a sparsity regularized Autoencoder for fusion of spatial and temporal data. We investigate Convolutional and Autoencoder Neural Networks for the extraction of spatial, temporal and fused spatio-temporal features and the subsequent application of kernel based Support Vector Machines for the binary detection of goal events. A high level overview of the processing pipeline is shown in Figure 1. While in fully-connected networks each hidden activation is computed by multiplying the entire input by the corresponding weights in that layer, in CNNs each hidden activation is computed by multiplying a small local input against the weights. The typical structure of a CNN consists of a number of convolution and pooling/subsampling layers, optionally followed by fully connected layers. At each convolution layer, the outputs of the previous layer are convolved with learnable kernels and passed through the activation function to form this layer’s output feature map. Let n× n be a square region extracted from a training input image X ∈ RN×M , and w be a filter of kernel size (m×m). The output of the convolutional layer h ∈ R(n−m+1)×(n−m+1) is given by: hi j = σ (m−1 ∑ a=0 m−1 ∑ b=0 wabx (i+a)( j+b)+b ` i j ) , (1) where b is the additive bias term, and σ(·) stands for the neuron’s activation unit. Specifically, the activation function σ , is a standard way to model a neuron’s output, as a function of its input. Convenient choices for the activation function include the logistic sigmoid, the hyperbolic tangent, and the Rectified Linear Unit. Taking into consideration the training time required for the gradient descent process, the saturating (i.e tanh, and logistic sigmoid) non-linearities are much slower than the non-saturating ReLU function. The output of the convolutional layer is directly utilized as input to a sub-sampling layer that produces downsampled versions of the input maps. There are several types of pooling, two common types of which are max-pooling and average-pooling, which partition the input image into a set of non-overlapping or overlapping patches and output the maximum or average value for each such sub-region. For the 2D feature extraction networks, we consider the VGG-16 CNN architecture, which is composed of 13 convolutional layers, with five of them being followed by a max-pooling layer, leading to three fully connected layers [9]. Unlike image detection problems, feature extraction in video must address the challenges associat

Abstract: After the sound, 2D images and videos, 3D models represented by polygonal meshes are the actual emergent content due to the technological advance in terms of 3D acquisition. 3D meshes can be subject to several degradations due to acquisition, compression, pre-treatment or transmission that distort the 3D mesh and therefore affect its visual rendering. Because the human observer is generally located at the end of this line, quality assessment of the content is required. We propose in this paper a view-independent 3D Blind Mesh Quality Assessment Index (BMQI) based on the estimation of visual saliency and roughness. Given a 3D distorted mesh, the metric can assess the percieved visual quality without the need of the reference content as humans do. No assumption on the degradation to evaluate is required for this metric, which makes it powerful and usable in any context requiring quality assessment of 3D meshes. Obtained results in terms of correlation with subjective human scores of quality are important and highly competitive with existing full-reference quality assessment metrics.

Abstract: Recent advances in computational models in vision science have considerably furthered our understanding of human visual perception. At the same time, rapid advances in convolutional deep neural networks (DNNs) have resulted in computer vision models of object recognition which, for the first time, rival human object recognition. Furthermore, it has been suggested that DNNs may not only be successful models for computer vision, but may also be good computational models of the monkey and human visual systems. The advances in computational models in both vision science and computer vision pose two challenges in two different and independent domains: First, because the lat-est computational models have much higher predictive accuracy, and competing models may make similar predictions, we require more human data to be able to statistically distinguish between different models. Thus we would like to have methods to acquire trustworthy human behavioural data fast and easy. Second, we need challenging experiments to ascertain whether models show similar input-output behaviour only near “ceiling” performance, or whether their performance degrades similar to human performance: only then do we have strong evidence that models and human observers may be using similar features and processing strategies. In this paper we address both challenges.

Abstract: In this paper, we train independent linear decoder models to estimate the perceived quality of images. More specifically, we calculate the responses of individual non-overlapping image patches to each of the decoders and scale these responses based on the sharpness characteristics of filter set. We use multiple linear decoders to capture different abstraction levels of the image patches. Training each model is carried out on 100,000 image patches from the ImageNet database in an unsupervised fashion. Color space selection and ZCA Whitening are performed over these patches to enhance the descriptiveness of the data. The proposed quality estimator is tested on the LIVE and the TID 2013 image quality assessment databases. Performance of the proposed method is compared against eleven other state of the art methods in terms of accuracy, consistency, linearity, and monotonic behavior. Based on experimental results, the proposed method is generally among the top performing quality estimators in all categories.

Abstract: Finding an objective image quality metric that matches the subjective quality has always been a challenging task. We propose a new full reference image quality metric based on features extracted from Convolutional Neural Networks (CNNs). Using a pre-trained AlexNet model, we extract feature maps of the test and reference images at multiple layers, and compare their feature similarity at each layer. Such similarity scores are then pooled across layers to obtain an overall quality value. Experimental results on four state-of-the-art databases show that our metric is either on par or outperforms 10 other state-of-the-art metrics, demonstrating that CNN features at multiple levels are superior to handcrafted features used in most image quality metrics in capturing aspects that matter for discriminative perception. c © 2016 Society for Imaging Science

Abstract: We formulate PRNU-based image manipulation localization as a probabilistic binary labeling task in a flexible discriminative random field (DRF) framework. A novel local discriminator based on the deviation of the measured correlation from the expected local correlation as estimated by a correlation predictor is paired with an explicit pairwise model for dependencies between local decisions. Experimental results from the Dresden Image Database indicate that the DRF outperforms prior art with Markov random field label priors.

Abstract: It is meaningful to detect outliers in traffic data for traffic management. However, this is a massive task for people from large-scale database to distinguish outliers. In this paper, we present two methods: Kernel Smoothing Na\"ive Bayes (NB) method and Gaussian Mixture Model (GMM) method to automatically detect any hardware errors as well as abnormal traffic events in traffic data collected at a four-arm junction in Hong Kong. Traffic data was recorded in a video format, and converted to spatial-temporal (ST) traffic signals by statistics. The ST signals are then projected to a two-dimensional (2D) (x,y)-coordinate plane by Principal Component Analysis (PCA) for dimension reduction. We assume that inlier data are normal distributed. As such, the NB and GMM methods are successfully applied in outlier detection (OD) for traffic data. The kernel smooth NB method assumes the existence of kernel distributions in traffic data and uses Bayes' Theorem to perform OD. In contrast, the GMM method believes the traffic data is formed by the mixture of Gaussian distributions and exploits confidence region for OD. This paper would address the modeling of each method and evaluate their respective performances. Experimental results show that the NB algorithm with Triangle kernel and GMM method achieve up to 93.78% and 94.50% accuracies, respectively.

Abstract: Immersive Analytics investigates how novel interaction and display technologies may support analytical reasoning and decision making. The Immersive Analytics initiative of Monash University started early 2014. Over the last few years, a number of projects have been developed or extended in this context to meet the requirements of semi- or full-immersive stereoscopic environments. Different technologies are used for this purpose: CAVE2™ (a 330 degree large-scale visualization environment which can be used for educative and scientific group presentations, analyses and discussions), stereoscopic Powerwalls (miniCAVEs, representing a segment of the CAVE2 and used for development and communication), Fishtanks, and/or HMDs (such as Oculus, VIVE, and mobile HMD approaches). Apart from CAVE2™ all systems are or will be employed on both the Monash University and the University of Konstanz side, especially to investigate collaborative Immersive Analytics. In addition, sensiLab extends most of the previous approaches by involving all senses, 3D visualization is combined with multi-sensory feedback, 3D printing, robotics in a scientific-artistic-creative environment.

Abstract: Plant phenotyping, or the measurement of plant traits such as stem width and plant height, is a critical step in the development and evaluation of higher yield biofuel crops. Phenotyping allows biologists to quantitatively estimate the biomass of plant varieties and therefore their potential for biofuel production. Manual phenotyping is costly, time-consuming, and errorprone, requiring a person to walk through the fields measuring individual plants with a tape measure and notebook. In this work we describe an alternative system consisting of an autonomous robot equipped with two infrared cameras that travels through fields, collecting 2.5D image data of sorghum plants. We develop novel image processing based algorithms to estimate plant height and stem width from the image data. Our proposed method has the advantage of working in situ using images of plants from only one side. This allows phenotypic data to be collected nondestructively throughout the growing cycle, providing biologists with valuable information on crop growth patterns. Our approach first estimates plant heights and stem widths from individual frames. It then uses tracking algorithms to refine these estimates across frames and avoid double counting the same plant in multiple frames. The result is a histogram of stem widths and plant heights for each plot of a particular genetically engineered sorghum variety. In-field testing and comparison with human collected ground truth data demonstrates that our system achieves 13% average absolute error for stem width estimation and 15% average absolute error for plant height estimation.

Abstract: A visual system cannot process everything with full fidelity, nor, in a given moment, perform all possible visual tasks. Rather, it must lose some information, and prioritize some tasks over others. The human visual system has developed a number of strategies for dealing with its limited capacity. This paper reviews recent evidence for one strategy: encoding the visual input in terms of a rich set of local image statistics, where the local regions grow — and the representation becomes less precise — with distance from fixation. The explanatory power of this proposed encoding scheme has implications for another proposed strategy for dealing with limited capacity: that of selective attention, which gates visual processing so that the visual system momentarily processes some objects, features, or locations at the expense of others. A lossy peripheral encoding offers an alternative explanation for a number of phenomena used to study selective attention. Based on lessons learned from studying peripheral vision, this paper proposes a different characterization of capacity limits as limits on decision complexity. A general-purpose decision process may deal with such limits by “cutting corners” when the task becomes too complicated. Human vision is full of puzzles. Observers can grasp the essence of a scene in less than 100 ms, reporting with a fair degree of reliability whether it is a beach or a street, whether it contains any animals, and what materials are present [1, 2]. Yet when probed for details, they are at a loss. Change the scene while masking the motion transients, and the observer may have great difficulty determining what has changed, even when the change is quite visible once it has been spotted (“change-blindness”, [3, 4]). Human vision is better than the best computer vision systems ever created, yet it is also easily fooled by visual illusions. People can look at a line drawing of a 3D object, and effortlessly understand its shape, yet have difficulty noticing the impossibility of an Escher never-ending staircase. We have difficulty finding our keys, even when they prove quite visible once found and fixated, and look nothing like other items on our desk. How does one explain this combination of marvelous successes and quirky failures? It perhaps seems unsurprising that these diverse phenomena at present have no unifying explanation. What do they have in common? Certainly, scene perception, object recognition, and 3-D shape estimation require different mechanisms at some stage of visual processing. Nonetheless, might there exist a coherent explanation in terms of a critical stage of processing that determines performance for a wide variety of tasks, or at least a guiding principle for what tasks are easy and difficult? Attempts to provide a unifying account have explained the failures in terms of the visual system having limited capacity (see [5] for a review). Our senses gather copious amounts of data, seemingly far more than our minds can fully process at once. At any given instant, we are consciously aware of only a small fraction of the incoming sensory input. We seem to have a limited capacity for awareness, for memory, and for the number of tasks we can simultaneously carry out, leading to poor performance at tasks that stress the capacity limits of the system. A classic example of the limited capacity logic concerns visual search. Suppose a researcher runs an experiment in which observers must find a target item among a number of other “distractor” items. As in many such experiments, the experimenter picks a target and distractors such that individual items seem easy to distinguish. Nonetheless, the researcher finds that search is inefficient, i.e., that it becomes significantly slower as one adds more distractors. Why is search difficult? One can easily discriminate the target from the distractors when looking directly at them. The poor search performance implies that vision is not the same everywhere, or, as Julian Hochberg put it, “vision is not everywhere dense” [6]. If vision were the same throughout the visual field, search would be easy. By a popular account, the main reason vision is not the same everywhere has to do with attention, in particular selective attention. In this account, attention is a limited resource, and vision is better where the observer attends than where they do not. The visual system deals with limited capacity by serially shifting attention. Some tasks require selective attention, and as a result are subject to the performance limits inherent in having to wait for this limited resource. In the case of difficult search tasks, for instance, the targetdistractor discrimination is presumed to require attention, making search significantly slower with increasing number of display items. On the other hand, preattentive tasks do not require attention; they can be performed quickly and in parallel, leading to easy search. Selective attention is typically described as a mechanism that gates access to further visual processing [7, 8, 9] rather than engaging in processing itself. Once the visual system selects a portion of the visual input, perception happens. Throughout this paper, when I refer to selective attention, I mean a gating mechanism. Traditionally, researchers have taken visual search phenomena as evidence that selective attention operates early in the visual processing pipeline, and that correct binding of basic features into an object requires selective attention [10]. Though this account has had a certain amount of predictive power when it comes to visual search, it has been problematic overall [11, 12, 13, 14]. The need for selective attention to bind basic features seems to conflict with: the relative ease of searching for a cube among differently lit cubes [15, 16, 17]; with easy extraction of the gist of a scene [18, 19, 2, 20, 21, 22, 23] and of ensemble properties of sets [24, 25, 26]; and with what tasks require attention in a dual-task paradigm [27]. My lab has argued instead that a main way in which the visual system deals with limited capacity is through encoding its inputs in a way that favors foveal vision over peripheral. Peripheral vision is, as a rule, worse than foveal vision, and often much worse. Peripheral vision must condense a mass of information into a succinct representation that nonetheless carries the information needed for vision at a glance. Only a finite number of nerve fibers can emerge from the eye, and rather than providing uniformly mediocre vision, the eye trades off sparse sampling in the periphery for sharp, high resolution foveal vision. This economical design continues into the cortex: more cortical resources are devoted to processing central vision at the expense of the periphery. We have proposed that the visual system deals with limited capacity in part by representing its input in terms of a rich set of local image statistics, where the local regions grow — and the representation becomes less precise — with distance from fixation [28]. Such a summary-statistic representation would render vision locally ambiguous in terms of the phase and location of features. Thus, this scheme trades off computation of sophisticated image features at the expense of spatial localization of those features. One of the main implications of this theory for vision science has been the need to re-examine understanding of visual attention. Most experiments investigating selective attention have had a peripheral vision confound. A number of phenomena previously attributed to attention may instead arise in large part from peripheral encoding. This paper begins by reviewing both phenomena in peripheral vision and our model of peripheral encoding. It reviews what we have learned about perception, as well as the implications for theories of attention, particularly selective attention. Our understanding of peripheral vision constrains possible additional mechanisms for dealing with limited capacity. In particular, I propose that the brain may face limits on decision complexity, and deal with those limits by performing a simpler version of any toocomplex task, leading to poorer performance at the nominal task. A lossy encoding in peripheral vision Peripheral vision is susceptible to clutter, as evidenced by the phenomena of visual crowding. Classic crowding refers to greater difficulty identifying a peripheral target when flanked by neighboring stimuli than when it appears in isolation. Crowded stimuli may appear jumbled and uncertain, lacking crucial aspects of form, almost as if they have a textural or statistical nature [29]. Crowding has often been studied with a target identification task, and with a target object flanked by other objects, but it almost certainly affects perception more generally. Crowding points to significant qualitative differences between foveal and peripheral vision. These differences are far greater than the modest differences between foveal and peripheral acuity, and are likely task-relevant for a wide variety of tasks [30]. The phenomena of crowding have been described in detail in a number of recent review papers [31, 32, 33, 34]. My lab has argued that one must control for or otherwise account for the strengths and limitations of peripheral vision before considering explanations based upon visual attention [30, 14, 35]. Otherwise, one risks fundamental misunderstandings about both perception and attention. Whether the paradigm is visual search, change detection, dual-task, scene perception, or inattentional blindness – all tasks whose results have been interpreted in terms of the mechanisms of attention – the often-cluttered stimuli lie at least in part outside of the fovea, and are potentially subject to crowding. A number of researchers have suggested that crowding results from “forced texture perception,” in which information is pooled over sizeable portions of the visual field [29, 36, 31, 32]. Based on these intuit

Abstract: Line segmentation performs a significant stage in the OCR systems; it has a direct effect on the character segmentation stage which affects the recognition rate. In this paper a robust algorithm is proposed for line segmentation for Arabic printed text system with and without diacritics based on finding the global maximum peak and the baseline detection. The algorithm is tested for different font sizes and types and results have been obtained from testing 5 types of fonts with total of 43,055 lines with 99.9 % accuracy for text without diacritics and 99.5% accuracy for text

Abstract: The implementation of automatic, adaptive filters in consumer imaging devices represents challenges to sharpness and resolution evaluation. The widely used e-SFR and other methods based on sine-waves and line targets are not necessarily representative of the capture of natural scene information. The recent dead leaves target is aimed at producing texture-MTFs that describe the capture of image detail under automatic non-linear, and content aware processes. A newer approach to the texture-MTFmeasurement that substitutes the dead leaves target with pictorial images is presented in this paper. The aim of the proposed method is to measure effective-MTFs indicative of system characteristics for given scenes and camera processes. Nine pictorial images, portraying a variety of subjects and textures, were set as targets for a DSLR camera and a high-end smartphone camera. Computed MTFs were found to be congruent with the dead leaves MTF. Scene dependency was reported mainly for the smartphone camera measurements, providing insight into the performance of the content-dependent processes. Results from the DLSR camera images, captured with minimum non-adaptive operations, were reasonably consistent for the majority of the scenes. Based on variations in scene-dependent MTFs, we make recommendations for scene content that is best for texture-MTF analysis.

Abstract: Humans resolve the spatial alignment between two visual stimuli at a resolution that is substantially finer than the spacing between the foveal cones. In this paper, we analyze the factors that limit the information at the cone photoreceptors that is available to make these acuity judgments (Vernier acuity). We use open-source software, ISETBIO1 to quantify the stimulus and encoding stages in the front-end of the human visual system, starting with a description of the stimulus spectral radiance and a computational model that includes the physiological optics, inert ocular pigments, eye movements, photoreceptor sampling and absorptions. The simulations suggest that the visual system extracts the information available within the spatiotemporal pattern of photoreceptor absorptions within a small spatial (0.12 deg) and temporal (200 ms) regime. At typical display luminance levels, the variance arising from the Poisson absorptions and small eye movements (tremors and microsaccades) both appear to be critical limiting factors for Vernier acuity.

Abstract: Glare due to sunlight, moonlight, or other light sources can be a serious impediment during autonomous or manual driving. Automatically detecting the presence, location, and severity of such glare can be of critical importance for an autonomous driving system, which may then give greater priority to other sensors or cues/parts of the scene. We present an algorithm for automatic real-time glare detection that uses a combination of: (1) the intensity, saturation, and local contrast of the input frame; (2) shape detection; and (3) solar azimuth and elevation computed based on the position and heading information from the GPS (used under daylight conditions). These data are used to generate a glare occurrence map that indicates the center location(s) and extent(s) of the glare region(s). Testing on a variety of daytime and nighttime scenes demonstrates that the proposed system is effective at glare detection and is capable of real-time operation. Introduction A very common occurrence during navigation is the presence of overly intense light that can overwhelm the driver or visual sensors of an autonomous ground vehicle, and thereby reduce visibility. When dazzled by a light source, typical reactions by the drivers are (1) flipping down the sunshades, which is not always effective, especially when the glare is caused by a low-positioned external light source, e.g., low sun or direct light from the headlights of the oncoming vehicle; (2) using one hand to occlude the strong light source from the eyes of the driver, which is not safe and can block the view of some important obstacles on the road; and (3) wearing sunglasses, which has comfort issues [1]. For autonomous vehicles which depend on onboard cameras for navigation, none of these reactions are possible, thus resulting in the partial or full loss of environment perception and subsequent failures. Although the detection of glare is an important problem, there are relatively few publications on glare detection using digital cameras [2, 3, 4, 5, 1, 6]. The existing approaches to glare detection generally rely on one or two simplistic image properties, followed by ad-hoc thresholding. The obvious and most commonly used feature is light intensity [2, 3], which can certainly detect the light source(s), but which cannot differentiate these sources from other bright regions (e.g., clouds), as demonstrated in Figure 1. To overcome this limitation, other approaches supplement intensity with hue, followed by spatial filtering or transformations [4, 5]. However, hue often fails to delineate the boundaries of the glare region(s), particularly when the image is subject to compression, as demonstrated in Figure 2. Instead of analyzing the scene, another class of approach has focused on analyzing an image of the drivers face [1, 6]. Unfortunately, this approach cannot be used in autonomous systems nor in scenarios where the drivers environment is different from the actual to-be-driven location (e.g., in remote navigation). Figure 1. Using light intensity alone to detect the glare regions also captures bright, non-glare regions: (a) Original image; (b) binarized intensity values using a threshold of 95% of the maximum value. Figure 2. HSV color space components of an image with glare: (a) Intensity values (V channel); (b) hue values (H channel); (c) saturation values (S channel). Previous approaches have used intensity and hue; part of our approach uses intensity and saturation, the latter being more reliable at detecting the spatial extent of each glare region. In this paper, we present an algorithm for glare detection that can reliably detect the presence of glare in both daytime and nighttime settings, and which can be used in autonomous or remote navigation settings (i.e., which does not rely on the analysis and co-presence of a human driver). Our approach employs an adaptive combination of features which remain computational simplistic, but which can reliably detect both the location(s) and extent(s) of the glare region(s). We specifically use a combination of: (1) the intensity, saturation, and local contrast of the input frame; (2) shape detection; and (3) solar azimuth and elevation computed based on the position and heading information from the GPS (used under daylight conditions). These data are used to generate a glare occurrence map that indicates the center location(s) and extent(s) of the glare region(s). The main contributions of this work are as follows: Our approach is the first to use a combination of photometric, geometric, and GPS information to perform glare detection. For the photoIS&T International Symposium on Electronic Imaging 2017 Autonomous Vehicles and Machines 2017 77 https://doi.org/10.2352/ISSN.2470-1173.2017.19.AVM-024 © 2017, Society for Imaging Science and Technology