Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
imagealgorithms:functionalities-new [2019/08/07 15:59]
pdufour [2. Image classification tools for data management]
imagealgorithms:functionalities-new [2019/08/21 14:58] (current)
pdufour [3.3. OCR]
Line 5: Line 5:
 ====== - Introduction ====== ====== - Introduction ======
  
-LTU delivers visual intelligence for image management and recognition tailored for your usecase.+LTU delivers visual intelligence ​solutions ​for image management and recognition tailored for your usecase.
 \\ \\
-So every case is considered individually. Depending on your need, LTU builds appropriate scenario calling multiple algorithms.\\ 
-You can choose to deploy on your own servers for security or privacy reasons, or use LTU's Cloud if you don’t want to deal with server purchasing and maintenance. 
 \\ \\
-This document explains you each technologies provided and used by LTU. +Every case is considered individually. Depending on your need, LTU builds appropriate scenario calling multiple algorithms.  
 +These principal functionalities are packaged in LTU Engine, which provides the components necessary for creating and managing visual search applications,​ including JSON API and a comprehensive Administrative Interface.  
 +Each technologies provided and used by LTU is explained in this document 
 +\\ 
 +\\ 
 +The solution is available via licensed software or via the hosted platform. You can choose to deploy on your own servers for security or privacy reasons, or use LTU's Cloud if you don’t want to deal with server purchasing and maintenance.
 \\ \\
 \\ \\
 {{ :​imagealgorithms:​scenario2.png?​nolink&​800 |}} {{ :​imagealgorithms:​scenario2.png?​nolink&​800 |}}
 \\ \\
-====== - Images ​structuration ​ ​======+ 
 +====== - Images ​management ​ ​======
 {{ :​imagealgorithms:​ltu-cl3.png?​nolink&​1200 |}} {{ :​imagealgorithms:​ltu-cl3.png?​nolink&​1200 |}}
 \\ \\
 \\ \\
-Organiser vos images ​en utilisant plusieurs techno ​+LTU Engine is a complete solution that will help you to organize and manage your images. All the following solutions could be used to structure and manage your images:  
--indexation -> creation DNA d'une image. Création base de données +  * Indexation: using to create your private database of searchable images 
--Deeplearning ​pour classification +  ​* ​Deeplearning: using to classify your images and to get associated keywords 
--Mot clés +  ​* ​OCR: using to get text informations from an image 
--OCR +  * Color algorythm: using to get colors of an image 
--couleur+ 
 +Once your images structured, you will be able to search inside by using keywords or/and image search algorythms. 
 + 
 +{{ :​imagealgorithms:​stolen_art2.png?​nolink&​700 |}}
 ===== - Image indexation ===== ===== - Image indexation =====
  
-LTU provides images recognition solutions that allows to find references from a query in a private images database. ​All that solutions are described in the second part of this whitepaper.+LTU provides images recognition solutions that allows to find references from a query in a private images database. ​The search ​solutions are described in the second part of this whitepaper.
 \\ \\
 +\\
 +{{ :​imagealgorithms:​algo-0094.png?​nolink&​1000 |}}
 \\ \\
 The first step toward making an image searchable is to create a descriptor of the image content. LTU Engine computes a visual signature for every image that describe its visual content in terms of color, shape, texture and many higher order visual features. These descriptors are also called image DNAs. The DNAs are unique for each image and specific to a search algorithm. The first step toward making an image searchable is to create a descriptor of the image content. LTU Engine computes a visual signature for every image that describe its visual content in terms of color, shape, texture and many higher order visual features. These descriptors are also called image DNAs. The DNAs are unique for each image and specific to a search algorithm.
 \\ \\
 \\ \\
-LTU Engine stores the DNAs in a database that constitutes your images reference database, in which you could make search by using LTU query retriaval solution. ​+{{gdraw>​1uaEUDl2BHGWP79LPzNyOMNmWeza7Iv52Zww8Z5QsSTA width=960 title="​reference database title" center}} 
 +\\ 
 + 
 +LTU Engine stores the DNAs in a database that constitutes your images reference database, in which you would make search by using LTU query retriaval solution.  
 +LTU Engine is fully optimized and let you index a collection of millions of images in a private database, store on a standard server/​computer and run all kind of queries on it in the twinkling of an eye.
 ===== - Artificial Inteligence ===== ===== - Artificial Inteligence =====
  
Line 46: Line 58:
 \\ \\
 \\ \\
-At each step, the "​bad"​ answers are eliminated and returned to the upstream levels to adjust the mathematical model. As you go along, the program reorganizes the information into more complex blocks. ​+At each step, the "​bad"​ answers are eliminated and returned to the upstream levels to adjust the mathematical model. As you go along, the program reorganizes the information into more complex blocks. ​The model is pre-trained on a training dataset. 
 +\\
 \\ \\
 When this model is subsequently applied to other cases, it is normally able to recognize an object without anyone ever telling him that he has never learned the concept of this object. When this model is subsequently applied to other cases, it is normally able to recognize an object without anyone ever telling him that he has never learned the concept of this object.
Line 54: Line 67:
 </​WRAP>​ </​WRAP>​
  
-Deep learning, can be optimized thanks to Transfer Learning: the knowledge acquired from the training dataset, called the "​source"​ dataset, is "​transferred"​ in order to properly handle the new dataset, named "​target"​. +Deep learning, can be optimized thanks to Transfer Learning: the knowledge acquired from the training dataset, called the "​source"​ dataset, is "​transferred"​ in order to properly handle the new dataset, named "​target"​. ​For example, knowledge gained while learning to recognize cars can be used to some extent to recognize trucks.
-\\+
 Thus, Transfer learning allows us to build accurate models in great time efficiency. ​ Thus, Transfer learning allows us to build accurate models in great time efficiency. ​
 \\ \\
 \\ \\
-Based on the Transfer Learning principles, LTU implements image classification tailored for specific needs. ​+Based on the Transfer Learning principles, LTU implements image classification tailored for your specific needs as described bellow
 \\ \\
  
Line 100: Line 112:
 ===== - Colors palette ===== ===== - Colors palette =====
  
-==== - Get image Colors ​====+==== - Image colors ​====
  
-LTU can return you the list of colors that are present in an image. ​+LTU Engine ​can return you the list of colors that are present in an image. ​
 \\  \\ 
 \\  \\ 
Line 108: Line 120:
 \\ \\
 \\ \\
-==== - Trend of an image collection ===== +==== - Trend of an images ​collection ===== 
-LTU Engine Color Query can analyze an image collection and return the most frequent colors. The set of the most frequent colors is what we call a color palette. ​+LTU Engine Color can analyze an images ​collection and return the most frequent colors. The set of the most frequent colors is what we call a color palette. ​
  
 The color palettes can be used to : The color palettes can be used to :
-  * suggest relevant queries to the user. (Queries that have results) +  * suggest relevant queries to the user 
-  * provide a quick overview on an image collection+  * provide a quick overview on an images ​collection
  
 An interesting feature of palette is that they can be computed on any subset of a collection. An interesting feature of palette is that they can be computed on any subset of a collection.
  
-For instance subsets can be categories. LTU Engine can compute a palette for the “Women Shirt” category. This will be different from the whole image collection palette. Some colors that are not present in this category will be removed and LTU Engine will introduce color nuances for the most present colors.+For instance subsets can be categories. LTU Engine can compute a palette for the “Women Shirt” category. This will be different from the whole images ​collection palette. Some colors that are not present in this category will be removed and LTU Engine will introduce color nuances for the most present colors.
  
 These subsets can also be result sets. If they are used to propose queries to the user, this feature can be a powerful tool for query refinement. These subsets can also be result sets. If they are used to propose queries to the user, this feature can be a powerful tool for query refinement.
Line 127: Line 139:
  
  
-====== - Image Recognition Technologies ​======+====== - Image recognition ​======
  
 {{ :​imagealgorithms:​ltu-re4.png?​nolink&​1200 |}} {{ :​imagealgorithms:​ltu-re4.png?​nolink&​1200 |}}
Line 137: Line 149:
 \\  \\ 
   * **[[functionalities#​visual_search_solutions|The Visual Search]]** that is divided into two recognition solutions – the image matching and the visual similarity search.   * **[[functionalities#​visual_search_solutions|The Visual Search]]** that is divided into two recognition solutions – the image matching and the visual similarity search.
-  * **[[functionalities#​fine_comparison_of_images|The Image Processing]]** that offers a Fine Images Comparison solution. +  * **[[functionalities#​fine_comparison_of_images|The Image Processing]]** that offers a Fine Images Comparison solution. ​
 \\ \\
-These principal functionalities are packaged in LTU Engine, which provides the components necessary for creating and managing visual search applications,​ including JSON API and a comprehensive Administrative Interface. ​ 
-\\  
-\\ 
-LTU Engine is fully optimized and let you index a collection of millions of images in a private database, store on a standard server/​computer and run all kind of queries on it in the twinkling of an eye. 
-\\ 
- 
-In this part, you are given details about: ​ 
-  ***[[functionalities#​image_matching_technology|Matching search]]** ​ 
-  ***[[functionalities#​similarity_search|Similarity search]]** 
-  ***[[functionalities#​color_search|Color search]]** 
-  ***[[functionalities#​fine_comparison_of_images|Fine Images Comparison]]** ​ 
 ===== - Visual Search Solutions ===== ===== - Visual Search Solutions =====
  
Line 162: Line 162:
 The visual search is composed by two key steps: The visual search is composed by two key steps:
  
-  ***Indexation**: ​The first step toward making an image searchable ​is to create a descriptor of the image content. LTU Engine computes a visual signature for every image that describe ​its visual content in terms of color, shape, texture and many higher order visual features ​and stores it in a reference database. These descriptors are also called image DNAs.+  ***Indexation**: ​[[functionalities#​image_indexation|As with the reference images]], the first step is to create a descriptor of the image content. LTU Engine computes a visual signature for every query that describes ​its visual content in terms of color, shape, texture and many higher order visual features. These descriptors are also called image DNAs.
  
   ***Retrivial**:​ A special comparison technology by which an image signature can be compared at extremely high speed with other image signatures from a database up to millions of images.   ***Retrivial**:​ A special comparison technology by which an image signature can be compared at extremely high speed with other image signatures from a database up to millions of images.
Line 184: Line 184:
  
 \\ \\
-{{ :​imagealgorithms:​distance.jpeg?​nolink&​800 |}}+{{ :​imagealgorithms:​algo-001.jpg?​nolink&​800 |}}
 ==== - Image Matching Technology ==== ==== - Image Matching Technology ====
 === - Overview === === - Overview ===
Line 241: Line 241:
   ***Brightness:​** Luminance settings correspond to the degree of luminance within each image pixel. ​ For a distant observer, the word ‘luminance’ is substituted by the word ‘brightness’,​ which corresponds to the sparkling parts of an object or image. ​   ***Brightness:​** Luminance settings correspond to the degree of luminance within each image pixel. ​ For a distant observer, the word ‘luminance’ is substituted by the word ‘brightness’,​ which corresponds to the sparkling parts of an object or image. ​
   ***Contrast:​** The difference between the darkest and the brightest parts of an image.   ***Contrast:​** The difference between the darkest and the brightest parts of an image.
-  ***Color change (Hue):​** ​ Changes in colorationhue is a complex color obtained by a mix of basic colors – Red, Blue, Green.+  ***Color change (Hue):​** ​ Changes in colorationhue is a complex color obtained by a mix of basic colors – Red, Blue, Green.
 \\  \\ 
 {{ :​imagealgorithms:​transformation.jpg?​nolink&​800 |}} {{ :​imagealgorithms:​transformation.jpg?​nolink&​800 |}}
Line 257: Line 257:
 == - Structural Transformations == == - Structural Transformations ==
  
-Structural transformations relate ​to changes that affect the structure of the image. These transformations do not limit the matching of images ​using LTU Engine’s image matching technology+Changes related ​to structural transformations ​affect the structure of the image. These transformations do not limit the matching of images. ​
 \\  \\ 
 \\ \\
Line 314: Line 314:
 === - Limitations === === - Limitations ===
  
-== - Structure modification too important == +== - Too important ​structure modification ​== 
-The examples below present challenges when matching, due to strong cropping with little structure or advanced composite images. ​+The examples below present challenges when matching, due to strong cropping with little structure or too advanced composite images. ​
 \\  \\ 
 \\  \\ 
Line 333: Line 333:
 The rate of the false positives depends on the application the image matching technology is integrated with. The rate of the false positives depends on the application the image matching technology is integrated with.
  
-Since image matching is very prone to detecting small common parts in images such as logos, it sometimes can result in false positives as seen below because parts of the images indeed ​did match.+Since image matching is very prone to detecting small common parts in images such as logos, it sometimes can result in false positives as seen below because parts of the images indeed match.
 \\  \\ 
 \\  \\ 
 {{ :​imagealgorithms:​fauxpositif.jpg?​nolink&​800 |}} {{ :​imagealgorithms:​fauxpositif.jpg?​nolink&​800 |}}
 \\ \\
-Sometimes image matching ​signature will detect ​the same object or scene, but not the same image. According to traditional image recognition ​research ​terminology these instances would be classified as false positives. However, these types of false positives are desirable when performing very fine similarity searches and when the objective is to match photographs taken of objects – proven especially relevant to mobile applications. ​+Sometimes image matching ​algorythm detects ​the same object or scene, but not the same image. According to traditional image recognition ​search ​terminology these instances would be classified as false positives. However, these types of false positives are desirable when performing very fine similarity searches and when the objective is to match photographs taken of objects – proven especially relevant to mobile applications. ​
 \\  \\ 
 \\  \\ 
Line 347: Line 347:
  
  
-Sometimes an image may not be indexed. This is due either to an unknown image format or due to missing image information, i.e. uniform ​colored images are rejected. ​Images containing very little information,​ i.e. having ​no distinct image features may be rejected too such as this image below. ​+Sometimes an image may not be indexed. This is due either to an unknown image format or due to missing image information 
 +  * Uniform ​colored images are rejected. ​ 
 +  * Having ​no distinct image features may be rejected too such as this image below. ​
 \\  \\ 
 \\ \\
 {{ :​imagealgorithms:​cafe2.png?​nolink&​400 |}} {{ :​imagealgorithms:​cafe2.png?​nolink&​400 |}}
    
-Finally images with dimensions less than 64x64 pixels are rejected in the default value of the LTU engine ​(the default setting can be changed).+Finally images with dimensions less than 64x64 pixels are rejected in the default value of the LTU Engine ​(the default setting can be changed).
  
 \\ \\
Line 366: Line 368:
  
 ==== - Similarity Search ==== ==== - Similarity Search ====
-=== - Overview ​===+=== - visual search ​===
  
-We provides a solution for finding similar images. By submitting a query image, our technology can find visually similar images.+LTU provides a solution for finding similar images. By submitting a query image, our technology can find visually similar images.
 \\ \\
 Similarity can focus on the shapes within the image, its color, or both to: Similarity can focus on the shapes within the image, its color, or both to:
Line 376: Line 378:
  
 \\ \\
-{{ :​imagealgorithms:​similarity.jpeg?​nolink&​1000 |}} 
-{{ :​imagealgorithms:​similarity2.jpeg?​nolink&​1000 |}} 
 \\ \\
 +{{ :​imagealgorithms:​algo-0092.jpeg?​nolink |}}
 \\ \\
-We recommended the advanced signature 4 for similarity search.+
 It analyzes two characteristics:​ shapes and colors. These parts are independent and their scores are only merged at the end into the final score of the signature. It analyzes two characteristics:​ shapes and colors. These parts are independent and their scores are only merged at the end into the final score of the signature.
  
Line 389: Line 390:
 On a finer level, the algorithm is able to detect the kind of texture used in the image. As a result, it finds paintings from the same painter to be similar, if the painter used the same texture techniques on different paintings. On a finer level, the algorithm is able to detect the kind of texture used in the image. As a result, it finds paintings from the same painter to be similar, if the painter used the same texture techniques on different paintings.
   ***Color:​**  ​   ***Color:​**  ​
-The color part of the signature ​is invariant to scale, rotation or any linear transformation. ​ Color search ​with signature 4 is quite flexible and can find images sharing the same colors. It also takes proportion of colors into account.+The color part of the signature is invariant to scale, rotation or any linear transformation. ​ Color search is quite flexible and can find images sharing the same colors. It also takes proportion of colors into account.
 \\ \\
 \\ \\
Line 397: Line 398:
   ***Color Weight 50:**  An intermediate value between zero and one hundred indicates that both shapes and colors should be taken into account.   ***Color Weight 50:**  An intermediate value between zero and one hundred indicates that both shapes and colors should be taken into account.
  
 +\\
  
- +=== - Search by keywords ​=== 
- +LTU solution also allows to search in an images collection by using one or a combination of several keywords. 
- +The similarity ​is not just visual anymore, it is thematic
- +For example, that could help to find all the pictures of a photograph whose theme is "​fruits"​
-==== - Limitations ==== +\\ 
-\\  +\\ 
-The shape part of the signature ​is currently sensible to scale and rotation+{{ :​imagealgorithms:​fruit.png?​nolink&​600 |}}
-Subsequent versions are expected ​to be invariant to scale. +
-\\  +
-\\  +
-These images illustrate just one example of the retrieval results possible with similarity search: +
-\\  +
-\\  +
-{{ :​imagealgorithms:​algo-008.jpg?​nolink&​1000 |}} +
 ==== - Color Search ==== ==== - Color Search ====
 === - Overview === === - Overview ===
Line 420: Line 414:
 {{ :​imagealgorithms:​colorsearch.png?​nolink&​800 |}} {{ :​imagealgorithms:​colorsearch.png?​nolink&​800 |}}
  
-LTU Engine Color Query is a powerful tool that analyzes ​the colors in an image. ​That allows to:+LTU Engine Color Query is a powerful tool that analyses ​the colors in an image. ​ 
 +As explained in the part 1 of this document, that allows to 
 +  * find the most popular color or color palette in a collection of images 
 +  * identify all colors in an image or collection of images: value and percentage 
 +But color could also be a criteria of search:
   * search for images by color(s) with optional color ponderation (e.g. 25% red, 75% green)   * search for images by color(s) with optional color ponderation (e.g. 25% red, 75% green)
-  * identify all colors in an image or collection of images: value and percentage 
   * upload an image to find images with similar colors   * upload an image to find images with similar colors
-  * find the most popular color or color palette in a collection of images 
  
 \\ \\
Line 431: Line 427:
  
  
-  + \\ 
-==== - Uniform Background removal ​====+=== - Uniform Background removal ===
  
 By default, the signature is computed on the whole image. On some specific case, this behavior can be problematic. For instance in eCommerce the articles are often shown on a uniform background. Thus the algorithm considers the background color as the article main color. By default, the signature is computed on the whole image. On some specific case, this behavior can be problematic. For instance in eCommerce the articles are often shown on a uniform background. Thus the algorithm considers the background color as the article main color.
 To tackle this issue, LTU Engine introduces a background removal algorithm that identifies uniform backgrounds and computes the signature only on the foreground image. If no uniform background is detected the signature is computed on the whole image. ​ To tackle this issue, LTU Engine introduces a background removal algorithm that identifies uniform backgrounds and computes the signature only on the foreground image. If no uniform background is detected the signature is computed on the whole image. ​
- 
-  
-Images to the right in these two examples show in blue the detected background:  ​ 
 \\  \\ 
 \\  \\ 
 {{ :​imagealgorithms:​algo-038.jpg?​nolink&​1000 |}} {{ :​imagealgorithms:​algo-038.jpg?​nolink&​1000 |}}
  
-  +\\ 
-==== - Queries ​====+=== - Queries ===
  
  
Line 451: Line 444:
 \\ \\
  
-=== - Query by color ===+== - Query by color ==
  
 With LTU Engine you can search in an image collection using a set of colors. With LTU Engine you can search in an image collection using a set of colors.
Line 477: Line 470:
 \\ \\
 \\ \\
-=== - Query by image ===+== - Query by image ==
  
  
Line 497: Line 490:
 ==== - Interaction with keywords ==== ==== - Interaction with keywords ====
  
-Keywords ​can be assigned to each image in the collection. Keywords can then be used to restrict the query result to some specific categories. For instance it is possible to run a query by color “red ​with keyword sofa”. Keywords are compatible with Query by color and Query by image.+ As explained in the part 1, keywords ​can be assigned to each image in collection. Keywords ​can be attributed manually or be the result of a deep learning process.\\ 
 +So, keywords ​can then be used with a visual search process ​to restrict the query result to some specific categories. For instance it is possible to run a query with keyword ​"sofa”. Keywords are compatible with Query by color and Query by image.
  
  
Line 510: Line 504:
 ===== - Link images with Metadata ===== ===== - Link images with Metadata =====
  
-LTU offers you to associate metadata to your images. So, once an image is recognized you can access to all data you have stored. Metadata are saved in a separated database.+LTU offers you to associate metadata to your images. So, once an image is recognized you can access to all data you have stored ​with. Metadata are saved in a separated database.
 {{gdraw>​15UmFRMYwtWyaB72kiybDRj7UD_fuZB6VG133T72BQgA width=940 title="​LTU Engine"​ center}} {{gdraw>​15UmFRMYwtWyaB72kiybDRj7UD_fuZB6VG133T72BQgA width=940 title="​LTU Engine"​ center}}
  
- 
-and optimizes the results returned by soliciting several image analysis algorithms. Once results are returned, this automated optimization loop tends to get as close as possible to the ground truth when the repository is poor quality or the queries are altered by exogenous factors. 
 \\ \\
  
 +===== - OCR =====
 +
 +LTU could use OCR algorythm to improve a result given by an image search.
 +
 +{{ :​imagealgorithms:​ocr-ex.png?​nolink&​600 |}}
  
-====== - Image data extraction ====== 
-Thus, LTU provides solutions to combine one or several technologies presented above to extract a maximum of informations. This part illustrates what kind of information it is possible to extract and with which technologies. 
  
-===== - Press & Fake News Monitoring ===== 
  
-{{ :​imagealgorithms:​press.png?​nolink&​700 |}} 
  
  
-===== - Insurance & Personal Property ===== 
-{{ :​imagealgorithms:​stolen_art2.png?​nolink&​700 |}} 
-\\ 
-===== - Education & Training ====== 
  
-{{ :​imagealgorithms:​education.png?​nolink&​700 |}} 
  
-===== - Retail & Consumer Apps ===== 
  
-{{ :​imagealgorithms:​product.png?​nolink&​700 |}} 
  
 ===== - Fine Comparison of Images ===== ===== - Fine Comparison of Images =====
Line 581: Line 567:
  
  
-  * Improve a result given by an image search 
- 
-{{ :​imagealgorithms:​ocr-ex.png?​nolink&​600 |}}