Announcement

Collapse
No announcement yet.

Some feedback on facial recognition

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Some feedback on facial recognition

    Although I am happy with facial recognition being introduced in 2019, I feel like the feature is not really designed for people with large family image collections in mind.

    So far my guess would be that the facial recognition gets it right for maybe 80-85% of my photos. When I tried Adobe a while back the hit rate was similar. With around 40,000 photos and a miss-rate of up to 15-20%, that means I have up to 6-8,000 photos I have to manually edit(!).

    What I find a bit strange is that the developers don't seem to realize that a lot of photo collections are of family photos, where people look alike for obvious reasons, and this confuses the software.

    A lot of the misses could be avoided or more easily correcte by some simple features:

    1. Let me include birth dates for each person. This way, my son will never be assigned to the photos of me or his elder brother as a baby, since my younger son wasn't born then (I get a lot of misses here).

    2. Only one instance of each person is allowed in each photo. Many misses happen because my older son is assigned to two different faces in the same photo (usually one of him and his brother).

    3. I don't know if the algorithms calculate some probable/approximate age of the person in a photo. However, this, combined with birthdates and date taken of the photo would be really helpful in deciding whether the photo is of me, my younger son or my older son.

    4. Right now I have to go through all of the photos because of the high miss rate. This is quite tedious. I would like some more tools so that I more quickly can assign the correct names to the photos, wihtout having to manually click through all of them to edit. For example, if I were presented with a score for each face of how (un)certain the algorithm is, I could start with the ones with lowest scores, because these are the most likely wrong.

    Don't get me wrong, I am happy that facial recognition is included in this edition; however, the software should be better adjusted to support the end user in correcting the cases which the algorithm gets wrong (which, in my case, are quite many in absolute numbers).

  • #2
    I would like to add that i agree with what Rockefoten says. I am happy to see facial recognition in ACDSee now, and look forward to its improvements!

    additionally, i would like to suggest to have a way to embed these face tags into IPTC/XMP, just like you can do with ACDSee Categories and ACDSee Keywords. I am currently about to start a painstaking process of scanning in thousands of photos and tagging them all with family member names, dates, etc, to then be shared with family members that probably do NOT have ACDSee, but some other photo software. Having all the facial recognition tags transferred to IPTC keywords would be extremely beneficial. Even better, would be for ACDSee to incorporate the new IPTC 2014/2017 extension standards that now includes a "Person(s) Shown" tag, rather than using the Core Keywords tag.

    Comment


    • #3
      Hello Everyone,

      Thank you for the feedback and please keep it coming. I am collecting all these ideas and soon will be having meetings with the dev team. We plan on improving Face Detection in an ACDSee 2019 update and ACDSee 2020.

      ACDSee 2019 is our 1st step.

      Regards,
      Mark

      Comment


      • #4
        Originally posted by mcosgrove View Post
        Thank you for the feedback and please keep it coming. I am collecting all these ideas and soon will be having meetings with the dev team. We plan on improving Face Detection in an ACDSee 2019 update and ACDSee 2020.
        I would like to +1 the need for results to be embedded in an commonly accessible tag. I love ACDSee, but I need metadata to be exposed so I can use that information in other apps and utilities (plus, it's the safest solution).

        I'm also unsure why detected but as-yet unnamed faces don't appear in the People Manager to be assigned names instead of having to find unnamed faces in individual photos.

        I am also seeing a tendency for ACDSee to assign the first name ever face tagged to fairly dissimilar faces that haven't yet been named. These should be detected and grouped as unknown individuals instead. Even when I've assigned that face the correct name, Redetect Faces will just put the same incorrect name back on the photo. Picasa really did this type of thing right, for the most part, so their workflow would be something to really use for inspiration.

        Lastly, since Picasa has been the only reasonable face tagging solution for the longest time (I found Lightroom's to be unmanageable for large collections), many users will have already invested numerous hours into tagging people in Picasa, so a conversion path from their tags is pretty important and could really help the adoption rate for ACDSee's face tagging (I really don't want to start all over again).

        Comment


        • #5
          Hi redJ,

          Thank you for all of your feedback.

          Regarding your third point about ACDSee assigning incorrect names, I have some workflows that may help:

          First, the best way to ensure accurate facial recognition is to first tag a few faces of each person you want recognized, and then allow Face Detection to start scanning more files. Facial Recognition will work better and better the more correct examples it has – essentially you need to “train” it a bit to recognize each person. This applies to any incorrect names you find too. The earlier you correct them, the less likely they are to influence recognition in future.

          If you notice a lot of incorrect names, one thing you can do is to use the Remove Faces and Redetect Faces commands in Manage mode:
          1. Select all the photos you can find that have incorrect names. I've found that searching each name and adding images to the Image Basket is a good way to do this.
          2. In the Tools menu, select Remove Faces to remove all faces and names from the selected images.
          3. Again in the Tools menu, select Redetect Faces to run Face Detection & Recognition on the selected images again.
          As long as you have a few correctly-named examples of each face in unselected images, this process should remove the incorrect names and rename the faces correctly. If you still find that this process leaves you with incorrect names, you may have missed some incorrect names, or you may need a few more correct examples.

          While the Redetect Faces command is also available in View mode, we originally intended it for cases where you want to run detection and recognition completely from scratch on a single image. So other images in your catalog will influence the recognition results, but not manual corrections to the image itself, which is why you're finding that names will change. If you're using Redetect Faces because of edits to images, you might find that the Rerun Face Detection option (in Tools > Options > Face Detection) gives you better results.

          Thanks,
          Tristan H.
          ACD Systems

          Comment


          • #6
            Originally posted by Tristan H View Post
            As long as you have a few correctly-named examples of each face in unselected images, this process should remove the incorrect names and rename the faces correctly. If you still find that this process leaves you with incorrect names, you may have missed some incorrect names, or you may need a few more correct examples.
            Thanks for your reply, Tristan.

            One of the things I mentioned is that ACDSee continued to assign the incorrect face even after I had made several correct identifications of that person (of photos in the exact same series/event - so very similar). What's implied here is that names ACDSee automatically (often incorrectly) assigns are given as much or more weight than manually assigned faces. That would further reinforce that Picasa's workflow is the correct way to handle face tagging. Picasa identifies who it believes a face belongs to and then asks for confirmation or correction (or groups faces it believes to be a new individual and asks for identification). In this way, Picasa gradually builds up it's ability to correctly identify (or exclude) persons from their identifications.

            Another frustrating aspect of the current ACDSee implementation that I'm running into is not getting name suggestions for already identified faces. It's pretty painful to be expected to manually type (or begin to) the name of every face detected in every photo. Often, selecting Redetect Faces will not detect any faces at all, even though an identical copy of the photo in the same folder does. For now, while ACDSee's face detection is new and evolving, I really think the biggest bang for the buck will be to allow users to access the face detection data embedded by Picasa and Lightroom. Until ACDSee gets their workflow and technology roughly on par with where Picasa was when it was discontinued, I have no plans on switching over to ACDSee's version, but it would be worth it if I could access and search the tens of thousands of already Picasa identified faces inside ACDSee.

            Lastly (for now), ACDSee seems to have no ability to detect faces that are not in the vertical orientation (unrotated photos, people reclining/lying down). I have photos where ACDSee refuses to detect a face that, if rotated 90 degrees, will then detect a face.

            Comment


            • #7
              The biggest issue's I have with Face Detection are;

              1 - We really need an 'unknown1' 'unknown 2' tag etc so that we can review where ACDsee has found a face and grouped them together but doesn't know the name - we can then simply work through the list of people ACDsee hasn't been able to name., pull out the errors and then just rename the 'tag' with the persons name and thus fix thousands potentially in one quick go.
              2 - Redetect Faces should have an option to only redetect auto-tagged faces and not the ones that we have manually corrected
              3 - The algorithm should give more weight to manually assigned faces than ones it has tagged itself when deciding on future tags.
              4 - In future it would be good to have the algorithm indicate how sure it is that it got it right so that we can quickly review say the 'red' - less than 50% match tags, or the amber 50-25% certain tag etc. that way we can more quickly correct significant errors and

              Comment


              • #8
                Originally posted by johnjbloomfield View Post
                4 - In future it would be good to have the algorithm indicate how sure it is that it got it right so that we can quickly review say the 'red' - less than 50% match tags, or the amber 50-25% certain tag etc. that way we can more quickly correct significant errors and
                Strongly endorse this. This is the biggest feature request I ever had for Picasa since bad matches inevitably creep into people sets, but are difficult to weed out after people have been matched to thousands of photos (but the false matches affect subsequent auto-match suggestions). It would be incredibly useful to be able to go back and look through low-confidence matches and remove the false-positives. This should even work for manually assigned matches (in Picasa, false positives not infrequently sneak in while assigning or confirming a large group of matches where one or two non matches are overlooked and confirmed with the set)

                Comment


                • #9
                  Originally posted by johnjbloomfield View Post
                  The biggest issue's I have with Face Detection are;

                  1 - We really need an 'unknown1' 'unknown 2' tag etc so that we can review where ACDsee has found a face and grouped them together but doesn't know the name - we can then simply work through the list of people ACDsee hasn't been able to name., pull out the errors and then just rename the 'tag' with the persons name and thus fix thousands potentially in one quick go.
                  2 - Re-detect Faces should have an option to only re-detect auto-tagged faces and not the ones that we have manually corrected
                  3 - The algorithm should give more weight to manually assigned faces than ones it has tagged itself when deciding on future tags.
                  4 - In future it would be good to have the algorithm indicate how sure it is that it got it right so that we can quickly review say the 'red' - less than 50% match tags, or the amber 50-25% certain tag etc. that way we can more quickly correct significant errors and
                  1 - I'm strongly supporting this - how else can I find people that have been detected but haven't got a name yet
                  2 - also strongly support this - I've had lots of people I had to correct and re-detection destroys this work
                  3 - ditto
                  4 - that would be nice

                  further a few more:
                  5 - I've had pictures with 7 times the same person detected, i.e. what Rockefoten mentioned in his point 2 is very important ... I've only found a single picture where a mirror made a person be shown twice
                  6 - I'd like to see a progress indicator ... the detection on all my photos took several days and at no point I had a clue how far the process was
                  7 - being able to create folders of people would be nice so I can create one for family, friends, work, others, unknown
                  8 - I would like to exclude pictures from detection, if you do street photography you otherwise have lots of people you're not interested in
                  9 - detection is too sensitive to lighting, I've had cases where I got different detection on the original raw and out of camera jpg of the same photo
                  10 - I had too many faces detected on things that are not people, e.g. rocks, trees, animals, etc. ... this requires some tuning in future releases
                  11 - before I ran detection on my image well I manually processed faces of a few common people ... result was that I had hundreds of wrongly assigned faces ... I almost got the impression that these faces were used almost as a kind of default ... also need improvement
                  12 - in View mode when editing assignments, when renaming a person I would be good to have to option to just rename that one picture vs. all people with that name (for this today I need to go to manage people first)
                  13 - I would like to be able to use people in searches, e.g. to say "all photos with people" or "photos with person X in London" (London being a keyword or category)
                  14 - Rockefoten's point 3 is also a key one ... I found lots of wrongly detected people where the name of a child was assigned to an elderly person and vice-versa, also the same name assigned to a mix of male and female

                  Comment


                  • #10
                    Originally posted by godspeedjas View Post
                    additionally, i would like to suggest to have a way to embed these face tags into IPTC/XMP
                    +1 from me

                    As far as I know, there are at least three possible sets of tags used by different programs to embed faces into photos:
                    * Windows Live Photo Gallery
                    * Picasa
                    * Adobe Photoshop
                    I don't think that another new tag should be invented.

                    ACDSee should be able to read all these formats and to use the data also for training its own algorithm.
                    ACDSee should also be able to write the data in one or all of these formats, configurably.

                    Comment


                    • #11
                      I have similar issures so the function is pretty useless for me how it works. I have assigned only a few news and now it seems this identity is assigned to many other people in my folders. Even if I correct one face ACDSee doesn't care about already assigned names and seems to stick to them unless I redetect the whole database.

                      It would be more intelligent to redetect those ones which were corrected. Example:
                      I have person A assigned
                      I add 2000 new photos with like 20 new faces. 12 of them are assigned to person A automatically. Bad detection rate
                      Now I correct the Name for person B in a few pictures (should be possible with Names List of the management window not only entering via keyboard.
                      But ACDSee is not learning that the detection of person A was wrong and with the new information is has to be corrected. How stupid honestly.

                      Well, for an alpha-version it is a funny tool. But I would like a usable tool I would pay money for. Too often I had to pay to be a "beta-tester" when buying the software. Well, some obvious and shameful mistakes still dragging along for 10 years. no one cares that aspect ration of watermarks is not kept when option of keeping it is used. Phewww.

                      Frustrated. Excuse me for that. There are many reasons. No other software I have this hate/love with.

                      Comment


                      • #12
                        I'd like to propose some additional workarounds to the face detection probabilities:

                        1. A manual option to assign a name to all the photos in a single sub-directory that have a single face detected. Not as elegant as a face detection algorithm that actually works, but until there is one...

                        2. A face-naming 'paint' tool in the Manage window or on another thumbnail display window, where the user could 'load' a name into a pointer and then either click on the photos to assign the name or within the photo on a face to assign the name to that face in the photo.

                        3. A hierarchical names tool tied to a pointer/'paint names tool' that when the user clicks on a face, opens a scrolling list of hierarchical classes of names (Names ->A->Z, either unlimited or at least 3-4 sub-classes per class. Scroll up and down, click and hold on a subclass which then opens a replacement list of the sub-class, and so on. (Easier to graphically illustrate than describe, I think...)

                        4. A face detection properties dialog, that displays what the detected parameters of the face is, so the user can adjust the parameters for auto-find.

                        5. A suggested workflow for detecting faces in a large collections of photos. One of the most pernicious issues in 2019 is that when the first face is named, the software assigns that name to nearly every face in the collection, regardless of whether it's a good match or not, causing lots of manual work later. Maybe put face detection on pause until the targeted collection of photos in how ever many subdirectories as selected have "enough" faces detected and named in each directory that the search has good data to work from.

                        Yes, all this is the result of having to resolve the issues ACDSee Ultimate has in parsing large collections with hundreds of possible faces. But they would make up for algorithmic deficiencies ad hardware limitations that make face detection so labor intensive today.

                        Comment


                        • #13
                          Drat, face recognition remains a alpha-level feature (?) in the 2019 update, at least for me. The photo archives I've used it against, remain tagged with the names of the first individual's name that I tagged, I may be missing something in the proposed workflow, but predominantly, if a face is identified, it's going to be the wrong person. Maybe a extended discussion regarding when to use the feature and when it's certain to fail, would be useful. While I can understand that developers don't wish to expose their algorithms' IP, more guidance seems necessary given the rather miserable performance of the software at this point. Microsoft's Live Picture Gallery, several years old, seem much more able and useful in recognizing faces in a general population. I'm hopeful there will be massive improvements, but given two releases that under-performed (again, in my collections) it seems worthwhile to discuss how, when and where the algorithms work, and plans for the future, to sustain hope in that future.

                          Comment


                          • #14
                            Hi AWH,

                            While we didn't make any changes to the Facial Recognition algorithm in the recent update, we did introduce a few tools that should make it much easier to get your photo collection properly labeled.

                            In Manage mode, Tools > Face Detection, there is a new "Rerun Recognition" command. When you select one or more photos, this command will clear names from all automatically-named faces in the selection, leaving the manually-named faces alone. Then it will rerun Facial Recognition on each unnamed detected face in the selection. Once you have a few manually labeled examples of each person, this command will help you to correct many of the incorrectly named faces.

                            The only issue with this tool is that if you miss an incorrectly named face and don't have it selected, it will continue to interfere. So we also added a new Rerun Recognition command in Tools > Options > Face Detection. This works exactly like the command in the menu, except that it runs on the entire database.

                            After running one of these commands, you will most likely find that there are still a few incorrectly named faces. Renaming a few of these manually and rerunning recognition again will make the set of labeled faces progressively more and more correct. In our tests, we found that even one pass significantly improved the number of correctly-labeled faces.

                            I hope that these new commands will help you to correct your photo collection more easily and quickly.

                            Thanks,
                            Tristan H.
                            ACD Systems

                            Comment

                            Working...
                            X