Hierarchical Cellular Automata for Visual Saliency

Transcription

1 Hierarchical Cellular Autoata for Visual Saliency Yao Qin Mengyang Feng 2 Huchuan Lu 2 Garrison W. Cottrell Received: 2 May 27 / Accepted: 26 Deceber 27 Springer Science+Business Media, LLC, part of Springer Nature 28 Abstract Saliency detection, finding the ost iportant parts of an iage, has becoe increasingly popular in coputer vision. In this paper, we introduce Hierarchical Cellular Autoata ) a teporally evolving odel to intelligently detect salient objects. consists of two ain coponents: Single-layer Cellular Autoata SCA) and Cuboid Cellular Autoata ). As an unsupervised propagation echanis, Single-layer Cellular Autoata can exploit the intrinsic relevance of siilar regions through interactions with neighbors. Low-level iage features as well as high-level seantic inforation extracted fro deep neural networks are incorporated into the SCA to easure the correlation between different iage patches. With these hierarchical deep features, an ipact factor atrix and a coherence atrix are constructed to balance the influences on each cell s next state. The saliency values of all cells are iteratively updated according to a well-defined update rule. Furtherore, we propose to integrate ultiple saliency aps generated by SCA at different scales in a Bayesian fraework. Therefore, single-layer propagation and ulti-scale integration are jointly odeled in our unified. Surprisingly, we find that the SCA can iprove all existing ethods that we applied it to, resulting in a siilar precision level regardless of the original results. The can act as an efficient pixel-wise aggregation algorith that can integrate state-of-the-art ethods, resulting in even better results. Extensive experients on four challenging datasets deonstrate that the proposed algorith outperfors state-of-the-art conventional ethods and is copetitive with deep learning based approaches. Keywords Saliency detection Hierarchical Cellular Autoata Deep contrast features Bayesian fraework Introduction Huans excel in identifying visually significant regions in a scene corresponding to salient objects. Given an iage, people can quickly tell what attracts the ost. In the field of coputer vision, however, perforing the sae task is very Counicated by Florent Perronnin. Yao Qin and Mengyang Feng have equally contributed. B Huchuan Lu lhchuan@dlut.edu.cn Yao Qin yaq7@eng.ucsd.edu Mengyang Feng engyangfeng@gail.co Garrison W. Cottrell gary@eng.ucsd.edu University of California, San Diego, CA, USA 2 Dalian University of Technology, Dalian, China challenging, despite draatic progress in recent years. To iic the huan attention syste, any researchers focus on developing coputational odels that locate regions of interest in the iage. Since accurate saliency aps can assign relative iportance to the visual contents in an iage, saliency detection can be used as a pre-processing procedure to narrow the scope of visual processing and reduce the cost of coputing resources. As a result, saliency detection has raised a great aount of attention Achanta et al. 29; Goferan et al. 2) and has been incorporated into various coputer vision tasks, such as visual tracking Mahadevan and Vasconcelos 29), object retargeting Ding et al. 2; Sun and Ling 2) and iage categorization Siagian and Itti 27; Kanan and Cottrell 2). Results in perceptual research show that contrast is one of the decisive factors in the huan visual attention syste Itti and Koch 2; Reinagel and Zador 999), suggesting that salient objects are ost likely in the region of the iage that significantly differs fro its surroundings. Many conventional saliency detection ethods focus on exploiting local and global con- 23

2 Stiulus GT DSR SCA MR Fig. An exaple illustrates that conventional saliency detection ethods based on handcrafted low-level features fail in coplex circustances. Fro top left to botto right: stiulus, Yan et al. 23), DSR Li et al. 23), MR Yang et al. 23), ground truth, Zhu et al. 24), and our ethod SCA and trast based on various handcrafted iage features, e.g., color features Liu et al. 2; Cheng et al. 25), focusness Jiang et al. 23c), textual distinctiveness Scharfenberger et al. 23), and structure descriptors Shi et al. 23). Although these ethods perfor well on siple bencharks, they ay fail in soe coplex situations where the handcrafted lowlevel features do not help salient objects stand out fro the background. For exaple, in Fig., the prairie dog is surrounded by low-contrast rocks and bushes. It is challenging to detect the prairie dog as a salient object with only lowlevel saliency cues. However, huans can easily recognize the prairie dog based on its category as it is seantically salient in high-level cognition and understanding. In addition to the liitation of low-level features, the large variations in object scales also restrict the accuracy of saliency detection. An appropriate scale is of great iportance in extracting the salient object fro the background. One of the ost popular ways to detect salient objects of different sizes is to construct ulti-scale saliency aps and then aggregate the with pre-defined functions, such as averaging or a weighted suation. In ost existing ethods Wang et al. 26; Li and Yu 25;Lietal.24a; Zhou et al. 24; Borjietal.25), however, these constructed saliency aps are usually integrated in a siple and heuristic way, which ay directly liit the precision of saliency aggregation. To address these two obvious probles, we propose a novel ethod naed Hierarchical Cellular Autoata ) to extract the salient objects fro the background efficiently. A Hierarchical Cellular Autoata consists of two ain coponents: Single-layer Cellular Autoata SCA) and Cuboid Cellular Autoata ). First, to iprove the features, we use fully convolutional networks Long et al. 25) to extract deep features due to their successful application to seantic segentation. It has been deonstrated that deep features are highly versatile and have stronger representational power than traditional handcrafted features Krizhevsky et al. 22; Farabet et al. 23; Girshick et al. 24). Low-level iage features and high-level saliency cues extracted fro deep neural networks are used by an SCA to easure the siilarity of neighbors. With these hierarchical deep features, the SCA iteratively updates the saliency ap through interactions with siilar neighbors. Then the salient object will naturally eerge fro the background with high consistency aong siilar iage patches. Secondly, to detect ulti-scale salient objects, we apply the SCA at different scales and integrate the with the based on Bayesian inference. Through interactions with neighbors in a cuboid zone, the integrated saliency ap can highlight the foreground and suppress the background. An overview of our proposed is shown in Fig. 2. Furtherore, the Hierarchical Cellular Autoata is capable of optiizing other saliency detection ethods. If a saliency ap generated by one of the existing ethods is used as the prior ap and fed into, it can be iproved to the state-of-the-art precision level. Meanwhile, if ultiple saliency aps generated by different existing ethods are used as initial inputs, can naturally fuse these saliency aps and achieve a result that outperfors each ethod. In suary, the ain contributions of our work include: ) We propose a novel Hierarchical Cellular Autoata to adaptively detect salient objects of different scales based on hierarchical deep features. The odel effectively iproves all of the ethods we have applied it to to stateof-the-art precision levels and is relatively insensitive to the original aps. 2) Single-layer Cellular Autoata serve as a propagation echanis that exploits the intrinsic relevance of siilar regions via interactions with neighbors. 3) Cuboid Cellular Autoata can integrate ultiple saliency aps into a ore favorable result under the Bayesian fraework. 2 Related Work 2. Salient Object Detection Methods of saliency detection can be divided into two categories: top-down task-driven) ethods and botto-up data-driven) ethods. Approaches like Alexe et al. 2; Marchesotti et al. 29; Ng et al. 22; Yang and Yang 22) are typical top-down visual attention ethods that require supervised learning with anually labeled ground truth. To 23

3 Iage Slicer Prior Maps Hierarchical Cellular Autoata SLIC SCA Updating Rule Updating Rule Final Saliency Map Feature Extractor Segentation Results Stiuli FCN Deep Features Constructed Graph Constructed Graph Ground Truth Fig. 2 The pipeline of our proposed Hierarchical Cellular Autoata. First, the stiulus is segented into ulti-scale superpixels, and superpixels on the iage boundary are selected as seeds for the propagation of the background Sect. 3.). Then FCN-32s Long et al. 25) isused as a feature extractor to obtain deep features Sect. 3.2). The generated prior aps and deep features are both fed into the Single-Layer Cellular Autoata Sect. 3.3.) to create ulti-scale saliency aps. Finally, we integrate these saliency aps via the Cuboid Cellular Autoata Sect ) to obtain our ultiate result better distinguish salient objects fro the background, highlevel category-specific inforation and supervised ethods are incorporated to iprove the accuracy of saliency aps. In contrast, botto-up ethods usually concentrate on lowlevel cues such as color, intensity, texture and orientation to construct saliency aps Hou and Zhang 27; Jiang et al. 2; Klein and Frintrop 2; Sun et al. 22; Tong et al. 25; Yan et al. 23). Soe global bottoup approaches tend to build saliency aps by calculating the holistic statistics on uniqueness of each eleent over the whole iage Cheng et al. 25; Perazzi et al. 22; Bruce and Tsotsos 25). As saliency is defined as a particular part of an iage that visually stands out copared to their neighboring regions or the rest of iage, one of the ost used principles, contrast prior, easures the saliency of a region according to the color contrast or geodesic distance against its surroundings Cheng et al. 23, 25; Jiang et al. 2; Jiang and Davis 23; Klein and Frintrop 2; Perazzi et al. 22; Wang et al. 2). Recently, the boundary prior has been introduced in several ethods based on the assuption that regions along the iage boundaries are ore likely to be the background Jiang et al. 23b; Li et al. 23; Wei et al. 22; Yang et al. 23; Borji et al. 25; Shen and Wu 22), although this takes advantage of photographer s bias and is less likely to be true for active robots. Considering the connectivity of regions in the background, Wei et al. 22) define the saliency value for each region as the shortest-path distance towards the boundary. Yang et al. 23) use anifold ranking to infer the saliency score of iage regions according to their relevance to boundary superpixels. Furtherore, in Jiang et al. 23a), the contrast against the iage border is used as a new regional feature vector to characterize the background. However, one of the fundaental probles with all these conventional saliency detection ethods is that the features used are not representative enough to capture the contrast between foreground and background, and this liits the precision of saliency detection. For one thing, low-level features cannot help salient objects stand out fro a lowcontrast background with siilar visual appearance. Also, the extracted global features are weak in capturing seantic inforation and have uch poorer generalization copared to the deep features used in this paper. 2.2 Deep Neural Networks Deep convolutional neural networks have recently achieved a great success in various coputer vision tasks, including iage classification Krizhevsky et al. 22; Szegedy et al. 25), object detection Girshick et al. 24; Hariharan et al. 24; Szegedy et al. 23) and seantic segentation Long et al. 25; Pinheiro and Collobert 24). With the rapid developent of deep neural networks, researchers have begun to construct effective neural networks for saliency detection Zhao et al. 25; Li and Yu 25; Zou and Koodakis 25; Wang et al. 25; Li et al. 26; Ki and Pavlovic 26). In Zhao et al. 25), Zhao et al. propose a unified ulti-context deep neural network taking both global and local context into consideration. Li et al. Li and Yu 25) and Zou et al. Zou and Koodakis 25) explore high-quality visual features extracted fro DNNs to iprove the accuracy of saliency detection. DeepSaliency in Li et al. 26) is a ulti-task deep neural network using a collaborative feature learning schee between two correlated tasks, saliency detection and seantic segentation, to learn better feature representation. One leading factor for the success of deep neural networks is the powerful expressibility and 23

4 strong capacity of deep architectures that facilitate learning high-level features with seantic inforation Hariharan et al. 25; Maetal.25). In Donahue et al. 24), Donahue et al. point out that features extracted fro the activation of a deep convolutional network can be repurposed to any other generic tasks. Inspired by this idea, we use the hierarchical deep features extracted fro fully convolutional networks Long et al. 25) to represent saller iage regions. The extracted deep features incorporate low-level features as well as highlevel seantic inforation of the iage and can be fed into our Hierarchical Cellular Autoata to easure the siilarity of different iage patches. 2.3 Cellular Autoata Cellular Autoata are a odel of coputation first proposed by Von Neuann 95). They can be described as a teporally evolving syste with siple construction but coplex self-organizing behavior. A Cellular Autoaton consists of a lattice of cells with discrete states, which evolve in discrete tie steps according to specific rules. Each cell needs to ake a decision on its next state in order to survive in the environent. How to ake a better decision? The sipliest way is to hold the current state forever. However, it is not wise as there will be no iproveents. Intuitively, we can see the nearest neighbors states as a reference. If we are siilar, then we should have siilar states; otherwise, we should have different states. Therefore, at each tie step, each cell intends to ake a wise decision for its next state based on its current state as well as its neighbors. Cellular Autoata have been applied to siulate the evolution of any coplicated dynaical systes Batty 27; Chopard and Droz 25; Cowburn and Welland 2; Aleida et al. 23; Martins 28; Pan et al. 26). Considering that salient objects are spatially coherent, we introduce Cellular Autoata into this field as an unsupervised propagation echanis. For saliency detection, we treat the saliency ap as a dynaic syste and the saliency values will be considered as the cell s states. The saliency value will evolve as tie goes by in order to get a better saliency ap, in other words, to ake the dynaic syste ore stable. Because ost salient objects in the iages share siilar feature representations as well as siilar saliency values, we propose Single-layer Cellular Autoata to exploit the intrinsic relationships of neighboring eleents of the saliency ap and eliinate gaps between siilar regions. Furtherore, it can be observed that there is a high contrast between salient objects and its surrounding backgrounds in the feature space. Through interacting with neighbors, it is easy for the dynaic syste to differentiate the foreground and background. In addition, we propose a ethod to cobine ultiple saliency aps generated by different algoriths, or cobine saliency aps at different scales through what we call Cuboid Cellular Autoata ). In, states of the autoaton are deterined by a cuboid neighborhood corresponding to autoata at the sae location as well as their adjacent neighbors in different saliency aps. An illustration of the idea is in Fig. 3b. In this setting, the saliency aps are iteratively updated through interactions aong neighbors in the cuboid zone. The state updates are deterined through Bayesian evidence cobination rules. Variants of this type of approach have been used before Rahtu et al. 2; Xie and Lu 2; Xie et al. 23; Li et al. 23). Xie et al. 23) use the low-level visual cues derived fro a convex hull to copute the observation likelihood. Li et al. 23) construct saliency aps through dense and sparse reconstruction and propose a Bayesian algorith to cobine the. Using Bayesian updates to cobine saliency aps puts the algorith for Cuboid Cellular Autoata on a fir theoretical foundation. 3 Proposed Algorith In this paper, we propose an unsupervised Hierarchical Cellular Autoata ) for saliency detection, coposed of two sub-units, a Single-layer Cellular Autoata SCA), and a Cuboid Cellular Autoata ), as described below. First, we construct prior aps of different scales with superpixels on the iage boundary chosen as the background seeds. Then, hierarchical deep features are extracted fro fully convolutional networks Long et al. 25) to easure the siilarity of different superpixels. Next, we use SCA to iteratively update the prior aps at different scales based on the hierarchical deep features. Finally, a is used to integrate the ulti-scale saliency aps using Bayesian evidence cobination. Figure 2 shows an overview of our proposed ethod. 3. Background Priors Recently, there have been various atheatical odels proposed to generate a coarse saliency ap to help locate potential salient objects in an iage Tong et al. 25a;Zhu et al. 24; Gong et al. 25). Even though prior aps are effective in iproving detection precision, they still have several drawbacks. For exaple, a poor prior ap ay greatly liit the accuracy of the final saliency ap if it incorrectly estiates the location of the objects or classifies the foreground as the background. Also, the coputational tie to construct a prior ap can be excessive. Therefore, in this paper, we build a quite siple and tie-efficient prior ap that only provides the propagation seeds for, which is quite insensitive to the prior ap and is able to refine this coarse prior ap into an iproved saliency ap. 23

5 First, we use the efficient Siple Linear Iterative Clustering SLIC) algorith Achanta et al. 2) to segent the iage into saller superpixels in order to capture the essential structural inforation of the iage. Let s i R denote the saliency value of the superpixel i in the iage. Based on the assuption that superpixels on the iage boundary tend to have a higher probability of being the background, we assign a close-to-zero saliency value to the boundary superpixels. For others, we assign a unifor value as their initial saliency values, s i = {. i boundary.5 i / boundary. Considering the great variation in the scales of salient objects, we segent the iage into superpixels at M different scales, which are displayed in Fig. 2 Prior Maps). 3.2 Deep Features fro FCN As is well-known, the features in the last layers of CNNs encode seantic abstractions of objects, and are robust to appearance variations, while the early layers contain low-level iage features, such as color, edge, and texture. Although high-level features can effectively discriinate the objects fro various backgrounds, they cannot precisely capture the fine-grained low-level inforation due to their low spatial resolution. Therefore, a cobination of these deep features is preferred copared to any individual feature ap. In this paper, we use the feature aps extracted fro the fully-convolutional network FCN-32s Long et al. 25)) to encode object appearance. The input iage to FCN-32s is resized to 5 5, and a -pixel padding is added to the four boundaries. Due to subsapling and pooling operations in the CNN, the outputs of each convolutional layer in the FCN fraework are not at the sae resolution. Since we only care about the features corresponding to the original iage, we need to ) crop the feature aps to get rid of the padding; 2) resize each feature ap to the input iage size via the nearest neighbor interpolation. Then each feature ap can be aggregated using a siple linear cobination as: gr i, r j ) = L l= ) ρ l df l i df l j 2, 2) where dfi l denotes the deep features of superpixel i on the l-th layer and ρ l is a weighting of the iportance of the l-th feature ap, which we set by cross-validation. The weights are constrained to su to : L l= ρ l =. Each superpixel is represented by the ean of the deep features of all contained pixels. The coputed gr i, r j ) is used to easure the siilarity between superpixels. 3.3 Hierarchical Cellular Autoata Hierarchical Cellular Autoata ) is a unified fraework coposed of single-layer propagation Single-layer Cellular Autoata) and ulti-scale aggregation Cuboid Cellular Autoata). It can generate saliency aps at different scales and integrate the to get a fine-grained saliency ap. We will discuss SCA and respectively in Sects and Single-Layer Cellular Autoata In Single-layer Cellular Autoata SCA), each cell denotes a superpixel generated by the SLIC algorith. SLIC takes the nuber of desired superpixels as a paraeter, so by using different nubers of superpixels with SCA, we can obtain aps at different scales. In this section, we assue one scale, denoted. We represent the nuber of superpixels in scale as n, but we oit the subscript in ost notations in this section for clarity, e.g., F for F, C for C and s for s. Different superpixel scales are treated independently. We ake three ajor odifications to the previous cellular autoata odels Sith 972; Von Neuann 95) for saliency detection. First, the states of cells in ost existing Cellular Autoata odels are discrete Von Neuann et al. 966; Wolfra 983). However, in this paper, we use the saliency value of each superpixel as its state, which is continuous between and. Second, we give a broader definition of the neighborhood that is siilar to the concept of z-layer neighborhood here z = 2) in graph theory. The z- layer neighborhood of a cell includes adjacent cells as well as those sharing coon boundaries with its adjacent cells. Also, we assue that superpixels on the iage boundaries are all connected to each other because all of the serve as background seeds. The connections between the neighbors are clearly illustrated in Fig. 3a. Finally, instead of unifor a) Fig. 3 The constructed graph odels used in our algorith. a Is used in SCA, the orange lines and the blue lines represent the connections between the blue center cell and its 2-layer neighbors. The purple lines indicate that superpixels on the iage boundaries are all connected to each other; b is used in, a cell e.g. the red pixel in the botto layer) is connected to the pixels with the sae coordinates in other layers as well as their four adjacent neighbors e.g. cells in blue color). All these pixels construct a cuboid interaction zone Color figure online) b) 23

6 influence of the neighbors, the influence is based on the siilarity between the neighbor to the cell in feature space, as explained next. Ipact Factor Matrix Intuitively, neighbors with ore siilar features have a greater influence on the cell s next state. The siilarity of any pair of superpixels is easured by a predefined distance in feature space. For the -th saliency ap, which has n superpixels in total, we construct an ipact factor atrix F R n n. Each eleent f ij in F defines the ipact factor of superpixel i to j as: { gr exp i,r j ) ) j NBi) f ij = σ 2 f j = i or otherwise, where gr i, r j ) is a function that coputes the distance between the superpixel i and j in feature space with r i as the feature descriptor of superpixel i. In this paper, we use the weighted distance of hierarchical deep features coputed by Eq. 2) to easure the siilarity between neighbors. σ f is a paraeter that controls the strength of siilarity and NBi) is the set of the neighbors of the cell i. In order to noralize the ipact factors, a degree atrix D = diag{d, d 2,...,d n } is constructed, where d i = j f ij. Finally, a row-noralized ipact factor atrix can be calculated as F = D F. Coherence Matrix Given that each cell s next state is deterined by its current state as well as its neighbors, we need to balance the iportance of these two factors. On the one hand, if a superpixel is quite different fro all its neighbors in the feature space, its next state will be priarily based on itself. On the other hand, if a cell is siilar to its neighbors, it should be assiilated by the local environent. To this end, we build a coherence atrix C = diag{c, c 2,...,c n } to proote the evolution aong all cells. Each cell s coherence towards its current state is initially coputed as: c i = ax f ij ),so it is inversely proportional to its axiu siilarity to its neighbors. As c i represents the coherence of the current state, we noralize it to be in a range c i [b, a + b ], where [ b, a + b ] [, ],via: ci c i in ) c j = a ax ) ) + b, 4) c j in c j where the in and ax are coputed over j =, 2,...,n. Based on preliinary experients, we epirically set the paraeters a and b in Eq. 4) to be and. The final, noralized coherence atrix is then: C = diag{c, c 2,..., c n }. Synchronous Update Rule In the existing Cellular Autoata odels, all cells will siultaneously update their states according to the update rule, which is a key point in Cellular Autoata, as it controls whether the ultiate evolving 3) Stiuli SCA Ground Truth Fig. 4 Saliency aps generated by SCA n = 2). The first three coluns show that salient objects can be precisely detected when the saliency appears in the center of the iage. The last three coluns indicate that SCA can still have good perforance even when salient objects touch the iage boundary state is chaotic or stable Wolfra 983). Here, we define the synchronous update rule based on the ipact factor atrix F R n n and coherence atrix C R n n : s t+) = C s t) + I C ) F s t), 5) where I is the identity atrix of diension n n and s t) R n denotes the saliency ap at tie t. When t =, s ) is the prior ap generated by the ethod introduced in Sect. 3.. AfterT S tie steps a tie step is defined as one update of all cells), the saliency ap can be represented as s T S). It should be noted that the update rule is invariant over tie; only the cells states s t) change over iterations. Our synchronous update rule is based on the generalized intrinsic characteristics of ost iages. First, superpixels belonging to the foreground usually share siilar feature representations. By exploiting the correlation between neighbors, the SCA can enhance saliency consistency aong siilar regions and develop a steady local environent. Second, it can be observed that there is a high contrast between the object and its surrounding background in feature space. Therefore, a clear boundary will naturally eerge between the object and the background, as the cell s saliency value is greatly influenced by its siilar neighbors. With boundarybased prior aps, salient objects can be naturally highlighted after the evolution of the syste due to the connectivity and copactness of the object, as exeplified in Fig. 4. Specifically, even though part of the salient object is incorrectly selected as the background seed, the SCA can adaptively increase their saliency values under the influence of the local environent. The last three coluns in Fig. 4 show that when the object touches the iage boundary, the results achieved by the SCA are still satisfying Cuboid Cellular Autoata To better capture the salient objects of different scales, we propose a novel ethod naed Cuboid Cellular Autoata ) to incorporate M different saliency aps generated 23

7 by SCA under M scales, each of which serves as a layer of the Cuboid Cellular Autoata. In, each cell corresponds to a pixel, and the saliency values of all pixels constitute the set of cells states. The nuber of all pixels in an iage is denoted as H. Unlike the definition of a neighborhood in Multi-layer Cellular Autoata in Qin et al. 25), where each pixel will only be influenced by the pixels with the sae coordination on other saliency aps, in, we enlarge the neighborhood into a cuboid zone. Here pixels with the sae coordinates in different saliency aps as well as their 4-connected pixels are all regarded as neighbors. That is, for any cell in a saliency ap, it should have 5M neighbors, constructing a cuboid interaction zone. The hierarchical graph is presented in Fig. 3b to illustrate the connections between neighbors. The idea that we want to enlarge the influential zone is inspired by the success of SCA, which indicates that the neighboring pixels will have a good ipact on the evolution of the saliency ap. In Cuboid Cellular Autoata, the saliency value of pixel i in the -th saliency ap at tie t is its probability of being the foreground F, represented as s t) = Pi t) F), while is its probability of being the background B, denoted s t) γ ), then it will be denoted as η t) =+. Correspondingly, η t) = eans that pixel i as s t) = Pi t) B). We binarize each ap with an adaptive threshold using Otsu s ethod Otsu 975), which is coputed fro the initial saliency ap and does not change over tie. The threshold of the -th saliency ap is denoted by γ.ifpixeli in the -th binary ap is classified as foreground at tie t s t) is binarized as background s t) <γ ). If pixel i belongs to the foreground, the probability that one of its neighboring pixels j in the -th binary ap is classified as foreground at tie t is denoted as P η t), j =+ i t) F ). In the sae way, the probability P η t), j = i t) B ) represents that the pixel j is binarized as B conditioned on that pixel i belongs to the background at tie t. We ake the siplifying assuption that P η t), j =+ i t) F ) is the sae for all the pixels in any saliency ap and it does not change over tie. Additionally, it is reasonable to assue that Pη t), j =+ i t) F ) = P η t) B ) if we siply consider to be, j = i t) the foreground and to be the background as two choices with the sae probability. Then we can use a constant λ to denote these two probablities: P η t), j =+ i t) F ) = P ηt), j = i t) B ) = λ. 6) Then the posterior probability Pi t) be calculated as follows: F η t), j =+) can P P ) i t) η F t), j =+ ) i t) F P = s t) λ ) Ω i t) F = η t), j =+ i t) F ) In order to get rid of the noralizing constant in Eq. 7), we define the prior ratio Ωi t) F) as: ) P i t) F P i t) B 7) ) = st) s t). 8) Cobining Eq. 7) and Eq. 8), the posterior ratio Ωi t) F η t), j =+) turns into: ) Ω i t) η F t), j =+ = P i t) P i t) = st) s t) As the posterior probability Pi t) F B η t) η t), j =+ ), j =+ ) λ λ. 9) F η t), j =+) represents the probability of pixel i of being the foreground F conditioned on that its neighboring pixel j in the -th saliency ap is binarized as foreground at tie t, Pi t) F η t), j =+) can also be used to represent the probability of pixel i of being the foreground F at tie t +. Then, s t+) = Pi t) F ηt), j =+). ) According to Eq. 9) and Eq. ), we can get: s t+) s t+) = = Pi t) Pi t) Pi t) Pi t) = st) s t) F η t), j =+) F η t), j =+) F η t), j =+) B η t), j =+) λ λ. ) It is uch easier to deal with the logarith of this quantity because the changes in logodds will be additive. So Eq. ) turns into: l s t+) ) ) = l s t) +, 2) ) ) where l s t+) t+) s = ln and = ln s t+) λ λ ) is a constant. The intuitive explanation for Eq. 2) is that: if 23

8 Stiuli SCA2 SCA4 SCA6 SCA8 SCA2 GT Fig. 5 Visual coparison of saliency aps generated by SCA at different scales n = 2, n 2 = 4, n 3 = 6, n 4 = 8 and n 5 = 2) and a pixel observes that one of its neighbors is binarized as foreground, it ought to increase its saliency value; otherwise, it should decrease its saliency value. Therefore, Eq. 2) can be turned into: l s t+) ) = l s t) ) + signs t) j,k γ k), 3) where s t) j,k is the saliency value of the pixel i s j-th neighbor in the k-th saliency ap at tie t and ust be greater than. In this paper, we epirically set =.4. As each pixel has 5M neighbors in total, the pixel will decide its action increase or decrease it saliency value) based on all its neighbors current states. Assuing the contribution of each neighbor is conditionally independent, we derive the synchronous update rule fro Eq. 3) as: l s t+) ) = l s t) ) + t), 4) where s t) R H is the -th saliency ap at tie t and H is the nuber of pixels in the iage. t) R H can be coputed by: t) = 5 M j= k= ) δk =, j > ) sign s t) j,k γ k, 5) where M is the nuber of different saliency aps, s t) j,k RH is a vector containing the saliency values of the j-th neighbor for all the pixels in the -th saliency ap at tie t and = [,,..., ] R H.Weuseδk =, j > ) to represent the occasion that the cell only has 4 neighbors instead of 5 in the -th saliency ap when it is in the -th saliency ap. After T C iterations, the final integrated saliency ap s T C ) is calculated by s T C ) = M M = s T C ). 6) In this paper, we use to integrate saliency aps generated by SCA at M = 5 scales. The five scales are Fig. 6 Coparison of saliency aps generated by different ethods and their optiized results via Single-layer Cellular Autoata. The first row is respectively input iages, ground truth and saliency aps generated by our proposed SCA with 2 superpixels. The second row displays original saliency aps generated by three traditional ethods fro left to right: CAS Goferan et al. 2), LR Shen and Wu 22), RC Cheng et al. 25)). The third row is their corresponding optiized results by SCA with 2 superpixels respectively, n = 2, n 2 = 4, n 3 = 6, n 4 = 8 and n 5 = 2. This cobination is denoted as, and the visual saliency aps generated by can be seen in Fig. 5.Here we use the notation SCAn to denote SCA applied with n superpixels. We can see that the detected objects in the integrated saliency aps are uniforly highlighted and uch closer to the ground truth. 3.4 Consistent Optiization 3.4. Single-Layer Propagation Due to the connectivity and copactness of the object, the salient part of an iage will naturally eerge with the Singlelayer Cellular Autoaton, which serves as a propagation echanis. Therefore, we use the saliency aps generated by several well-known ethods as the prior aps and refresh the according to the synchronous update rule. The saliency aps achieved by CAS Goferan et al. 2), LR Shen and Wu 22) and RC Cheng et al. 25)aretakenass ) in Eq. 5). The optiized results via SCA are shown in Fig. 6. We can see that the foreground is uniforly highlighted and a clear object contour naturally eerges with the autoatic single-layer propagation echanis. Even though the original saliency aps are not particularly good, all of the are significantly iproved to a siilar accuracy level after evolution. That eans our ethod is independent of prior aps and can ake a consistent and efficient optiization towards state-of-the-art ethods Pixel-Wise Integration A variety of ethods have been developed for visual saliency detection, and each of the has its advantages and liitations. As shown in Fig. 7, the perforance of a saliency 23

9 FT ITTI CAS Stiuli BL MR GT a) Stiulus FT-SCA ITTI-SCA CAS-SCA Stiuli MDF DS MCDL GT b) GT MCA * Fig. 7 Effects of pixel-wise saliency aggregation with Cuboid Cellular Autoata. We integrate saliency aps generated by three conventional algoriths: BL Tong et al. 25a), Yan et al. 23) and MR Yang et al. 23) ina) and incorporate saliency aps generated by three deep learning ethods: MDF Li and Yu 25), DS Li et al. 26), MCDL Zhao et al. 25) inb). The integrated result is denoted as.a Saliency aggregation of three conventional ethods. b Saliency aggregation of three deep learning ethods Fig. 8 Effects of holistic optiization by Hierarchical Cellular Autoata. We use MCA Qin et al. 25), and to integrate saliency aps generated by three classic ethods: FT Achanta et al. 29), ITTI Itti et al. 998) and CAS Goferan et al. 2). Their respective saliency aps optiized by SCA with 2 superpixels are shown in the second row. Note that * uses as input the saliency aps processed by SCA the second row) and applies to the, while the MCA and odels are applied directly to the first row detection ethod varies with individual iages. Each ethod can work well for soe iages or soe parts of the iages but none of the can perfectly handle all the iages. Furtherore, different ethods ay copleent each other. To take advantage of the superiority of each saliency ap, we use Cuboid Cellular Autoata to aggregate two groups of saliency aps, which are generated by three conventional algoriths: BL Tong et al. 25a), Yan et al. 23) and MR Yang et al. 23) and three deep learning ethods: MDF Li and Yu 25) anddslietal.26) and MCDL Zhao et al. 25). Each of the serves as a layer in Eq. 4). Figure 7 shows that our proposed pixel-wise aggregation ethod, Cuboid Cellular Autoata, can appropriately integrate ultiple saliency aps and outperfors each one. The saliency objects on the aggregated saliency ap are consistently highlighted and uch closer to the ground truth. of Cellular Autoata s ) SCA + = Here we show that when is applied to soe poor) prior aps, it does not perfor as well as when the prior ap is post-processed by SCA. This otivates their cobination into. As is shown in Fig. 8, when the candidate saliency aps are not well constructed, both and MCA Qin et al. 25) fail to detect the salient object. Unlike and MCA, overcoes this liitation through incorporating singlelayer propagation SCA) together with pixel-wise integration ) into a unified fraework. The salient objects can be intelligently detected by regardless of the original perforance of the candidate ethods. When we use to integrate existing ethods, the optiized results will be denoted as *. 4 Experients In order to deonstrate the effectiveness of our proposed algoriths, we copare the results on four challenging datasets: Yan et al. 23), MSRA5 Liu et al. 2), Li et al. 24b) and Li and Yu 25). The Extended Coplex Scene Saliency Dataset ) contains iages with ultiple objects of different sizes. Soe of the iages coe fro the challenging Berkeley-3 dataset. MSRA- 5 contains ore coprehensive iages with coplex background. The PASCAL- S dataset derives fro the validation set of PASCAL VOC2 Everingha et al. 2) segentation challenge and contains 85 natural iages. The last dataset, HKU- IS, contains 4447 challenging iages and their pixel-wise saliency annotation. In this paper, we use as the validation dataset to help choose the feature aps in FCN Long et al. 25). We copare our algorith with 2 classic or state-of-theart ethods including ITTI Itti et al. 998), FT Achanta et al. 29), CAS Goferan et al. 2), LR Shen and Wu 22), XL3 Xie et al. 23), DSRLi et al. 23), Yan et al. 23), UFO Jiang et al. 23c), MR Yang et al. 23), Jiang et al. 23b), Zhu et al. 24), RC Cheng et al. 25), HDCT Ki et al. 24), BL Tong et al. 25a), BSCA Qin et al. 25), Wang et al. 25), MCDL Zhao et al. 25), MDF Li and Yu 25), DSLi et al.26), SSD- Ki and Pavlovic 26), where 23

10 the last 5 ethods are deep learning-based ethods. The results of different ethods are either provided by authors or achieved by running available code or binaries. The code of will be publicly available at our project site and github. 4. Paraeter Setup For the Single-layer Cellular Autoaton, we set the nuber of iterations T S = 2. For the Cuboid Cellular Autoata, we set the nuber of iterations T C = 3. We deterined epirically that SCA and converge by 2 and 3 iterations, respectively. We choose M = 5 and run SCA with n = 2, n 2 = 4, n 3 = 6, n 4 = 8 and n 5 = 2 superpixels to generate ulti-scale saliency aps for. 4.2 Evaluation Metrics We evaluate all ethods by standard - PR) curves via binarizing the saliency ap with a threshold sliding fro to 255 and then coparing the binary aps with the ground truth. Specifically, precision = SF GF, recall = SF SF GF, 7) GF where SF is the set of the pixels segented as the foreground, GF denotes the set of the pixels belonging to the foreground in the ground truth, and refers to the nuber of eleents in a set. In any cases, high precision and recall are both required. These are cobined in the to obtain a single figure of erit, paraeterized by β: F β = + β 2 ) precision recall β 2 precision + recall 8) where β 2 is set to as suggested in Achanta et al. 29)to ephasize the precision. To copleent these two easures, we also use ean absolute error ) to quantitatively easure the average difference between the saliency ap s R H and the ground truth g R H in pixel level: = H H s i g i. 9) i= indicates how siilar a saliency ap is copared to the ground truth, and is of great iportance for different applications, such as iage segentation and cropping Perazzi et al. 22). In addition, we also copute the Area Under ROC Curve AUC) to better copare the perforance of different ethods. 4.3 Validation of the Proposed Algorith 4.3. Feature Analysis In order to construct the Ipact Factor atrix, we need to choose the features that will enter into Eq. 2). Here we analyze the efficacy of the features in different layers of a deep network in order to choose these feature layers. In deep neural networks, earlier convolutional layers capture fine-grained low-level inforation, e.g., colors, edges and texture, while later layers capture high-level seantic features. In order to select the best feature layers in the FCN Long et al. 25), we use as a validation dataset to easure the perforance of deep features extracted fro different layers. The function gr i, r j ) in Eq. 3) can be coputed as gr i, r j ) = dfi l dfj l, 2) 2 where dfi l denotes the deep features of superpixel i on the l-th layer. The outputs of convolutional layers, relu layers and pooling layers are all regarded as a feature ap. Therefore, we consider in total 3 layers of fully convolutional networks. We do not take the last two convolutional layers into consideration as their spatial resolutions are too low. We use the the higher, the better) and ean absolute error ) the lower, the better) to evaluate the perforance of different layers on the dataset. The results are shown in Fig. a and b. The score is obtained by thresholding the saliency aps at twice the ean saliency value. We use this convention for all of the subsequent results. The x-index in Fig. a and b refers to convolutional, ReLu, and pooling layers as ipleented in the FCN. We can see that deep features extracted fro the pooling layer in Conv and Conv5 can achieve the best two scores, and also perfor well on. The saliency aps in Fig. 9 correspond to the bars in Fig..Here it is visually apparent that salient objects are better detected with the final pooling layers of Conv and Conv5. Therefore, in this paper, we cobine the feature aps fro pool and pool5 with a siple linear cobination. Equation 2) then turns into: gr i, r j ) = ρ dfi 5 dfj ρ ) dfi 3 dfj 3, 2) 2 where ρ balance the weight of pool and pool Paraeter Learning We learn the paraeters in our via a grid search with the dataset as the validation set, e.g., ρ in Eq. 2), σ f in 23

11 Fig. 9 Visual coparison of saliency aps with different layers of deep features. The left two coluns are the input iages and their ground truth. Other coluns present the saliency aps with different layers of deep features. The color bars on the top stand for different convolutional layers see Fig. a, b) Conv Conv2 Conv3 Conv4 Conv Layer Layer a) MSRA b) Feasure & AUC a=.7 a= a= AUC 8 a=.7 a= 6 a=.5.75 a=.4 a= 4 a=.2 a= b a) Precison weighted suation pool5 pool c) Precison weighted suation pool5 pool Fig. a The score for each layer in FCN-32s on ; b the score for each layer in FCN-32s on ; c and d - curves of SCA using deep features extracted fro pool and pool5 as well as a weighted suation of these two layers of deep features. a bars, b bars, c and d - curves coparison Eq. 3), a and b in Eq. 4) and Λ in Eq. 2). The, AUC and scores of SCA2 are used for paraeter selection. We vary ρ fro to and plot the perforance versus ρ in Fig. a. We can see that when ρ = 25, the and AUC achieve the highest scores higher is better) and achieves the lowest value lower is better). Therefore, we epirically set ρ = 25 and apply it to all other datasets. For the paraeter σ f, we test the perforance of SCA2 when varies fro to 25. The plots of, AUC σ 2 f and are shown in Fig. b. It can be seen that the best perforance is achieved when = 7. Therefore, we use σ 2 f σ 2 f = 7 in all other experients. In our paper, we use the two hyperparaeters a and b to control c i [b, a+b], where [b, a+b] [, ].InFig.d, we copare the perforance of different cobinations of a and b. First, we choose a fro. to, taking an interval of.. Correspondingly, the paraeter b is also chosen fro. to with an interval of., constrained by a + b. d) Feasure & AUC Feasure & AUC.7 AUC / f b) AUC a= b a= b c) AUC a= a= a= a=.7 a= a=.5 a=.4 a= a=.2 Fig. Experients for the paraeter learning on the dataset. The, AUC and scores of SCA2 are used for paraeter selection. a ρ in SCA2. b σ f in SCA2. c in. d a and b in SCA2 We can see fro Fig. d that when a = and b =, the value is the sallest, AUC and scores are the highest. All the three evaluation etrics have the best scores when ci [.]. Therefore, we use the this cobination: a =, b = for all the experients in our paper. After fixing all the hyperparaeters discussed above ρ = 25, /σ 2 f = 7, a = and b = ), we conduct a grid search when the paraeter Λ in varies fro to.2. The plots of, AUC and scores of versus the paraeter Λ are shown in Fig. c. It is easy to see that as Λ becoes larger, the value becoes d) a= a= a= a=.7 a= a=.5 a=.4 a= a=.2 23

12 .4 -color SCA2-color SCA6-color SCA2-color.2 -deep SCA2-deep SCA6-deep SCA2-deep.4 -color SCA2-color SCA6-color SCA2-color.2 -deep SCA2-deep SCA6-deep SCA2-deep SCA2 SCA2 SCA4 SCA4.2.2 SCA6 SCA6 SCA8 SCA8. SCA2. SCA Fig. 2 Coparison between BSCA Qin et al. 25) that uses color featuers and our SCA using deep features Color figure online) saller better). However, the AUC score also decreases a bit. This is because that when Λ is large, the saliency ap integrated by will be uch closer to a binary ap. Then the value will becoe saller and AUC score Fig. 3 / curves of saliency aps generated by SCA at different scales n = 2, n 2 = 4, n 3 = 6, n 4 = 8 and n 5 = 2 respectively), and the integrated results by on and will becoe poorer. The score is not very sensitive to the paraeter Λ. To balance the perforance of and AUC score, we choose Λ =.4 as our final setting. ITTI.7 FT CAS LR XL3 DSR.5 UFO.4 MR.7 RC HDCT BL BSCA.5 MCDL MDF.4 DS SSD ITTI FT CAS LR.4 XL3 DSR.2 UFO MR RC HDCT BL.4 BSCA MCDL.2 MDF DS SSD MSRA5 MSRA5 MSRA5 MSRA5 ITTI.7 FT CAS LR XL3 DSR.5 UFO.4 MR.7 RC HDCT BL.5 BSCA.4 MDF SSD ITTI FT CAS LR.4 XL3 DSR.2 UFO MR RC.4 HDCT BL BSCA.2 MDF SSD ITTI.7 FT CAS LR XL3 DSR.5 UFO.4 MR.7 RC HDCT BL BSCA.5 MCDL MDF.4 DS SSD ITTI FT CAS LR.4 XL3 DSR.2 UFO MR RC HDCT BL.4 BSCA MCDL.2 MDF DS SSD ITTI FT CAS LR XL3.5 DSR.4 MR.7 RC HDCT BL BSCA.5 MCDL MDF.4 DS SSD a) ITTI FT CAS.4 LR XL3 DSR.2 MR RC HDCT BL.4 BSCA MCDL.2 MDF DS SSD b) Fig. 4 PR curves, FT curves and scores of different ethods copared with our algorith ). Fro top to botto:, MSRA5, and are tested. a PR curves. b FT curves 23

13 Table Coparison of AUC, and scores of 2 state-of-the-art ethods as well as our proposed on all four bencharks Method MSRA5 PASCALS AUC AUC AUC AUC ITTI FT CAS LR XL DSR UFO MR RC HDCT BL BSCA MCDL MDF DS SSD We ark the first, second, third results in bold, italic, bolditalic respectively. Deep ethods are annotated with underlines Coponent Effectiveness To test the effectiveness of the integrated deep features, we show the - curves of Single-layer Cellular Autoata with each layer of deep features as well as the integrated deep features on two datasets. The - curves in Fig. c and d deonstrate that hierarchical deep features outperfor single-layer features, as they contain both category-level seantics and fine-grained details. In addition, we copare the perforance between our SCA and the BSCA in Qin et al. 25) to see the superiority of deep features over low-level color features. The PR curves and / curves are displayed in Fig. 2. Here we copare the two SCAs at different scales and the perforance of on the dataset. It is notable to see that the deep features iprove the perforance with a large argin copared to using color features. In order to deonstrate the effectiveness of our proposed, we test the perforance of each coponent in on the standard and datasets. We generate saliency aps at five scales: n = 2, n 2 = 4, n 3 = 6, n 4 = 8, n 5 = 2 and use to integrate the. FT curves in Fig. 3 indicate that the results of the Single-layer Cellular Autoata are already quite satisfying. In addition, can iprove the overall perforance of SCA with a wider range of high scores than SCA alone. Siilar results are also achieved on other datasets but are not presented here to be succinct Perforance Coparison We display the - curves and /- curves of 2 state-of-art ethods as well as our proposed in Fig. 4 and the AUC, and scores in Table. As is shown in Fig. 4 and Table, our proposed Hierarchical Cellular Autoata perfors favorably against state-of-the-art conventional algoriths with higher precision and recall values on four challenging datasets. is copetitive with deep learning based approaches. The fairly low value indicates that our saliency aps are very close to the ground truth. As MCDL Zhao et al. 25) and DSLi et al. 26) trained the network on the MSRA dataset, we do not report their results on this dataset in Fig. 4 and Table.. In addition, Wang et al. 25) used part of the iages in the MSRA and datasets as the training set. As a result, we only test with the test 23

14 Stiuli GT Ours BL DSR MR MDF DS MCDL SSD- Fig. 5 Visual coparison of saliency aps of different ethods. GT: Ground Truth, Ours: Saliency aps generated by Hierarchical Cellular Autoata ) iages on these two datasets. Saliency aps are shown in Fig. 5 for visual coparison of our ethod with other odels. 4.4 Optiization of State-of-the-Art Methods In the previous sections, we showed qualitatively that our odel creates better saliency aps by iproving initial saliency aps with SCA, or by cobining the results of ultiple algoriths with, or by applying SCA and. Here we copare our ethods to other ethods quantitatively. When the initial aps are iperfect, we apply SCA to iprove the and then apply. When the initial aps are already very good, we show that we can cobine stateof-the-art ethods to perfor even better by siply using Consistent Iproveent In Sect. 3.4., we concluded that results generated by different ethods can be effectively optiized via Single-layer Cellular Autoata. Figure 6 shows the precision-recall curves and ean absolute error bars of various saliency ethods and their optiized results on four datasets. These results deonstrate that SCA can greatly iprove existing results to a siilar precision level. Even though the original saliency aps are not well constructed, the optiized results are coparable to the state-of-the-art ethods. It should be noted that SCA can even optiize deep learning-based ethods to a better precision level, e.g., MCDL Zhao et al. 25), MDF Li and Yu 25), Wang et al. 25), SSD- Ki and Pavlovic 26). In addition, for one existing ethod, we can use SCA to optiize it at different scales 23

15 DSR.5 BL LR UFO.4 MR.4 MCDL MDF RC BL DSR LR MR SSD- Models MSRA5 MSRA5.25 MSRA DSR.5 LR BL.4 MR.4 UFO MDF RC DSR.5 BL LR UFO.4 MR.4 MCDL MDF RC BL DSR LR MR SSD- Models BL DSR LR MR SSD- Models.7.5 DSR LR.5 BL.4 MR.4 MCDL RC BL DSR LR MR SSD- a) b) c) Models Fig. 6 Consistent iproveent of our proposed SCA and on four datasets. a) andb): PR curves of different ethods dashed line) and their optiized version via SCA2 solid line). The right colun shows that SCA2 triangle), iproves the s of the original ethods ultiplication sybol) and that * circle), here applied to SCA2, SCA6, and SCA2, further iproves the results. a PR curves for unsupervised odels. b PR curves for supervised odels. c scores and then use to integrate the ulti-scale saliency aps. The ultiate optiized result is denoted as *. The lowest s of saliency aps optiized by in Fig. 6c show that s use of iproves perforance over SCA alone. 23

16 .7.5 BL BL.2.4 MR MR DS DS MCDL.2 MCDL.4 MDF MDF BL BL.2.4 MR MR DS DS MCDL.2 MCDL.4 MDF MDF AUC BL MR AUC DS MCDL MDF AUC BL MR AUC DS MCDL MDF.7.5 BL BL.2.4 MR MR DS DS MCDL.2 MCDL.4 MDF MDF AUC BL MR AUC DS MCDL MDF a) b) c) Fig. 7 Effects of pixel-wise aggregation via Cuboid Cellular Autoata on, and dataset. For each dataset, the first row copares three conventional ethods BL Tong et al. 25a), Yan et al. 23), MR Yang et al. 23) andtheir integrated results via Cuboid Cellular Autoata, denoted as. The second row copares three deep learning odels, e.g. DS Li et al. 26), MCDL Zhao et al. 25), MDF Li and Yu 25)) and their integrated results. The precision, recall and scores in the right colun are obtained by thresholding the saliency aps at twice the ean saliency value. a PR curves. b FT curves. c Score bars 23

17 FT ITTI CAS MCA * MSRA FT ITTI CAS MCA * FT ITTI CAS MCA * FT ITTI CAS MCA * MSRA Models a) b) c) Fig. 8 Coparison between three different integration ethods MCA Qin et al. 25), and when integrating FT Achanta et al.29), ITTI Itti et al. 998) and CAS Goferan et al. 2) on and MSRA datasets. a PR curves. b FT curves. c scores MSRA Models Effective Integration In Sect , we used Cuboid Cellular Autoata as a pixel-wise aggregation ethod to integrate two groups of state-of-the-art ethods. One group includes three of the latest conventional ethods while the other contains three deep learning-based ethods. We test the various ethods on the, and datasets, and the integrated result is denoted as. PR curves in Fig. 7a strongly prove the effectiveness of that outperfors all the individual ethods. FT curves of in Fig. 7b are fixed at high values that are insensitive to the selective thresholds. In addition, we binarize the saliency ap with two ties ean saliency value. Fro Fig. 7c we can see that the integrated result has higher precision, recall and F- easure scores copared to each ethod that is integrated. Also, the ean absolute errors of are always the lowest as displays. The fairly low ean absolute errors indicate that the integrated results are quite siilar to the ground truth. Although Cuboid Cellular Autoata has exhibited great strength in integrating ultiple saliency aps, it has a ajor drawback that the integrated result highly relies on the precision of candidate saliency detection ethods as MCA in Qin et al. 25). If saliency aps fed into Cuboid Cellular Autoata are not well constructed, Cuboid Cellular Autoata cannot naturally detect the salient objects via ineractions between these candidate saliency aps., however, can easily address this proble as it incorporates single-layer propagation and ulti-scale integration into a unified fraework. Unlike MCA and, can achieve better integrated saliency ap regardless of their original detection perforance. PR curves, FT curves and scores in Fig. 8 show that ) has a better perforance than MCA as it considers the influence of adjacent cells at different scales. 2) can greatly iprove the aggregation results copared to MCA and because it is independent to the initial saliency aps. Siilar results are also achieved on other datasets but are not presented here to be succinct. 4.5 Run Tie The run tie is to process one iage in MSRA5 dataset via Matlab R24b-64bit with a PC equipped with an i7-479k 3.6 GHz CPU and 32GB RAM. Table 2 displays the average run tie of each coponent in our algorith except for the tie for extracting deep features. We can see that Single-layer Cellular Autoata and Cuboid Cellular Autoata are very fast to process one iage, on average 6 s. And the holistic takes only 582 s to process one iage without superpixel segentation and 324 s with SLIC. 23

18 Table 2 Run tie of each coponent of Method SCA2 SCA4 SCA6 SCA8 SCA2 w/ SLICs) wo/ SLICs) Table 3 Coparison of run tie Model Year Code Ties) Model Year Code Ties) Model Year Code Tie s) Matlab.768 HDCT 24 Matlab MR 23 Matlab.4542 MCDL 25 Python Matlab.484 XL3 23 Matlab Matlab + C Matlab 8.4 LR 22 Matlab.259 MDF 25 Matlab DSR 23 Matlab RC 2 C 6 BL 25 Matlab EXE 82 CA 2 Matlab + C Deep ethods are annotated with underlines Stiuli Ground Truth Fig. 9 Failure cases of our proposed We also copare the run tie of our ethod with other state-of-the-art ethods in Table 3. Here we copute the run tie including superpixel segentation and feature extraction for all odels. It can be observed that our algorith has the least run tie copared to other deep learning based ethods and is the top 5 fastest ethod aong all the ethods. 4.6 Failcure Cases We further qualitatively analyze soe failure cases with our proposed and present the saliency aps in Fig. 9. We can see that when a large aount of objects are touching the iage boundary, our fails to detect the whole objects. This is because we directly use the iage boundary as the background seeds in our algorith. We hypothesize that well-selected background seeds ay alleviate the proble to a great extent. How to efficiently and effectively select the background seeds is left as a future work. 5 Conclusion In this paper, we propose an unsupervised Hierarchical Cellular Autoata, a teporally evolving syste for saliency detection. With superpixels on the iage boundary chosen as the background seeds, Single-layer Cellular Autoata is designed to exploit the intrinsic connectivity of saliency objects through interactions with neighbors. Low-level iage features and high-level seantic inforation are both extracted fro deep neural networks and incorporated into SCA to easure the siilarity between neighbors. The saliency aps will be iteratively updated according to welldefined update rules, and salient objects will naturally eerge under the influence of neighbors. This context-based propagation echanis can iprove the saliency aps generated by existing ethods to a siilar perforance level with higher accuracy. In addition, Cuboid Cellular Autoata is proposed to aggregate ultiple saliency aps generated by SCA under different scales based on Bayesian fraework. Meanwhile, Cuboid Cellular Autoata and Hierarchical Cellular Autoata can act as a saliency aggregation ethod to incorporate saliency aps generated by ultiple state-of-art ethods into a ore discriinative saliency ap with higher precision and recall. Experiental results deonstrate the superior perforance of our algoriths copared to other existing ethods. Acknowledgeents MY Feng and HC Lu were supported in part by the National Natural Science Foundation of China under Grants , 6528, YQ and GWC were also partially supported by Guangzhou Science and Technology Planning Project Grant No ). References Achanta, R., Heai, S., Estrada, F., & Susstrunk, S. 29). Frequency-tuned salient region detection. In Proceedings of IEEE 23