CLIP

This API runs a general-purpose image classifier.
It compares each item in a list of prompts to each item in a list of image URLs.

Add this to your python code:

import booste

out_dict = booste.clip(api_key, prompts, images)

Arguments:

Arg Description Required Type Example
api_key Your API key, found on the User Dashboard True string "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
prompts A list of potential captions to assess against the image(s). True list [string] ["A photo of a dog.", "A basketball."]
images A list of images to assess against the prompt(s). Can be either URLs or local paths to the images. True list [string] ["https://upload.wikimedia.org/wikipedia/commons/7/7a/Basketball.png", "./my-dog.jpg"]
pretty_print Tells the client to print the output in an easily readable way to the terminal. False (default = False) Boolean True

Return Type:

Dict - a json blob of the following shape:

{
   "A photo of a dog":{
      "https://upload.wikimedia.org/wikipedia/commons/7/7a/Basketball.png":{
         "similarity":"0.21043204",
         "probabilityRelativeToPrompts":2.035336001426913e-05,
         "probabilityRelativeToImages":0.06019586697220802
      },
      "./my-dog.jpg":{
         "similarity":"0.23791271",
         "probabilityRelativeToPrompts":0.9932945966720581,
         "probabilityRelativeToImages":0.9398041367530823
      }
   },
   "A basketball":{
      "https://upload.wikimedia.org/wikipedia/commons/7/7a/Basketball.png":{
         "similarity":"0.31845447",
         "probabilityRelativeToPrompts":0.9999796748161316,
         "probabilityRelativeToImages":0.9999978542327881
      },
      "./my-dog.jpg":{
         "similarity":"0.18793154",
         "probabilityRelativeToPrompts":0.00670537818223238,
         "probabilityRelativeToImages":2.145161033695331e-06
      }
   }
}

Where the first keys are the prompts, and the nested keys are the image urls/paths.

"similarity" is a float value indicating how similar the prompt is to that image.

"probabilityRelativeToPrompts" is a float value indicating how probable that prompt is to that one image when compared against all other prompts.

"probabilityRelativeToImages" is a float value indicating how probable that image is to that one prompt when compared against all other images.

The probabilities can be hard to visually parse, so use the pretty_print = True argument to help understand the output.

---------
Similarity scores
---------


    Prompt:         A photo of a dog 
    Image URL:     https://upload.wikimedia.org/wikipedia/commons/7/7a/Basketball.png 
    Similarity:     0.21043204

    Prompt:         A photo of a dog 
    Image URL:     ./my-dog.jpg 
    Similarity:     0.23791271

    Prompt:         A basketball 
    Image URL:     https://upload.wikimedia.org/wikipedia/commons/7/7a/Basketball.png 
    Similarity:     0.31845447

    Prompt:         A basketball 
    Image URL:     ./my-dog.jpg 
    Similarity:     0.18793154


---------
Prompt probabilities for each image:
---------

    Image: https://upload.wikimedia.org/wikipedia/commons/7/7a/Basketball.png

        Prompt: A photo of a dog 
        Prob: 2.035336001426913e-05

        Prompt: A basketball 
        Prob: 0.9999796748161316

    Image: ./my-dog.jpg

        Prompt: A photo of a dog 
        Prob: 0.9932945966720581

        Prompt: A basketball 
        Prob: 0.00670537818223238


---------
Image probabilities for each prompt:
---------

    Prompt: A photo of a dog

        Image: https://upload.wikimedia.org/wikipedia/commons/7/7a/Basketball.png 
        Prob: 0.06019586697220802

        Image: ./my-dog.jpg 
        Prob: 0.9398041367530823

    Prompt: A basketball

        Image: https://upload.wikimedia.org/wikipedia/commons/7/7a/Basketball.png 
        Prob: 0.9999978542327881

        Image: ./my-dog.jpg 
        Prob: 2.145161033695331e-06