Survey on Interactive Visual Search
Keywords:
Mobile visual search, multimodal search, interactive search, mobile deviceAbstract
Mobile phones have evolved into powerful image processing devices equipped with high-resolution cameras, color displays, and hardware-accelerated graphics. This survey paper describes a novel multimodal interactive image search system on mobile devices. The system, the Joint search with ImaGe, Speech, And Word Plus (JIGSAW ), takes full advantage of the multimodal input and natural user inter-actions of mobile devices. It is designed for users who already have pictures in their minds but have no precise descriptions or names to address them. By describing it using speech and then refining the recognized query by interactively composing a visual query using exemplary images, the user can easily find the desired images through a few natural multimodal interactions with his/her mobile device.
References
[1] (2009).Google Goggles [Online]. Available: http://www.google.com/mobile/goggles/
[2] (2006). Nokia Point and Find [Online]. Available: https://en.wikipedia.org/wiki/Nokia_Point_%26_Find
[3] Kooaba. (2007). Kooaba [Online]. Available: http://www.kooaba.com
[4] B. Erol, E. Antúnez, and J. Hull, “Hotpaper: Multimedia interaction with paper using mobile phones,” in Proc. 16th ACM Multimedia Conf., New York, 2008.
[5] J. Graham and J. J. Hull, “Icandy: A tangible user interface for itunes,” in Proc. CHI ’08: Extended Abstracts on Human Factors in Computing Systems, Florence, Italy, 2008.
[6] J. J. Hull, B. Erol, J. Graham, Q. Ke, H. Kishi, J. Moraleda, and D. G. Van Olst, “Paper-based augmented reality,” in Proc. 17th Int. Conf. Artificial Reality and Telexistence (ICAT), Washington, DC, 2007.
[7] Amazon. (2007). SnapTell [Online]. Available: http://www.snaptell.com.
[8] ”Mobile visual search”, IEEE SIGNAL PROCESSING MAGAZINE [72] JULY 2011.
[9] Houqiang Li, Yang Wang, Tao Mei, Jingdong Wang, and Shipeng Li, “Interactive Multimodal Visual Search on Mobile Device”, IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 15, NO. 3, APRIL 2013.
[10] K. Church, B. Smyth, P. Cotter, and K. Bradley, “Mobile information access: A study of emerging search behavior on the mobile internet,” ACM Trans. Web, vol. 1, no. 1, May 2007.
[11] [Online].Available:http://www.pwc.com/gx/en/communications/review/features/mobile-data.jhtml
[12] Google.[Online]Available:http://www.google.com/
[13] TinEye.[Online].Available:http://www.tineye.com/
[14] GazoPa. [Online]. Available: http://www.gazopa.com/
[15] Y. Cao, H. Wang, C. Wang, Z. Li, L. Zhang, and L. Zhang, “MindFinder: Interactive sketch-based image search on millions of images,” in Proc. ACM Multimedia, 2010, pp. 1605–1608.
[16] M. Eitz, K. Hildebrand, T. Boubekeur, and M. Alexa, “Sketch-based image retrieval: Benchmark and bag-of-features descriptors,” IEEE Trans. Vis. Comput. Graphics, vol. 17, no. 11, pp. 1624–1636, Nov. 2011.
[17] M. Eitz,K.Hildebrand, T. Boubekeur, and M. Alexa, “An evaluation of descriptors for large-scale image retrieval from sketched feature lines,” Comput. Graphics, vol.34, no.5, pp. 482–498, 2010.
[18] H. Xu, J. Wang, X. Hua, and S. Li, “Image search by concept map,” in Proc. ACM SIGIR, 2010, pp. 275–282.
[19] T. Chen, M.-M. Cheng, P. Tan, A. Shamir, and S.-M. Hu, “Sketch2photo: Internet image montage,” ACM Trans. Graph., vol. 28, no. 5, pp. 124:1–124:10, Dec. 2009.
[20] Yojana S. Pawale,Prof. Prakash Devale, “I2MMS: An Interactive Multi Modal Visual Search Technique”, International Journal of Advanced Research in Computer and Communication Engineering Vol. 4, Issue 5, May2015
[21] A. Nodari ,M. Ghiringhelli, A. Zamberletti, M. Vanetti, S. Albertini, I. Gallo , “A mobile visual search application for content based image retrieval in the fashion domain” , IEEE JUNE 2012
[22] Anushma C R, "Multimodal Image Search System on Mobile Device", International Journal of Computer Organization Trends- Volume 7 Number 1- Apr 2014
[23] F. Jing, M. Li, H.-J. Zhang, and B. Zhang, “An efficient and effective region-based image retrieval framework,” IEEE Trans. Image Process., vol. 13, no. 5, pp. 699–709, 2004.
[24] X. Liu, Y. Lou, A.W. Yu, and B. Lang, “Search by mobile image based on visual and spatial consistency,” in Proc. IEEE Int. Conf.Multimedia
[25] V. Chandrasekhar, D. M. Chen, A. Lin, G. Takacs, S. S. Tsai, N. M.Cheung, Y. Reznik, R.Grzeszczuk, and B.Girod, “Comparison of local feature descriptors for mobile visual search,” in Proc. IEEE Int. Conf.Image Process., 2010, pp. 3885–3888
[26] Jiemin wang1,2,3, Yuanhai He3, Yujie Zhou3, Yu Qiao1,” iGAPSearch: Using Phone Cameras to Search Around the World”.
[27] [27]Digimarc Discover [Online]. Available: https://www.digimarc.com/discover/
[28] [28]LinkMe Mobile. [Online]. Available: http://www.snapnow.co.uk/
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Under this license, authors retain ownership of the copyright for their articles. By submitting to the International Journal of Advanced Research in Science, Management, and Technology (IJARSMT), authors grant the journal the right of first publication. Users are free to share, copy, and redistribute the material in any medium or format, and to adapt, remix, transform, and build upon the material for any purpose, including commercially, provided that appropriate credit is given to the original author(s) and the journal, a link to the license is provided, and any changes made are indicated.
