Thank you for sharing this wonderful project. I am curious about how do you implement clustering of face landmarks. Can you describe that in detail? Or can you sharing some related papers or projects? Thanks in advance.

Hi, @hysts

First of all, thank you so much for the great work!

I'm a graduate student and have used your pretrained model to generate landmark points as ground truth. I'm currently finishing up my thesis writing and want to cite your github repo.

I don't known if I overlooked something, but I couldn't find the citation information in the README page. Is there anyway to cite this repo?

Thank you.

Hi, the colab notebook looks broken. I used it about 2 weeks ago with out any problem. Basically in dependcie installing phase, when executing "mim install mmcv-full", colab will ask if I want to use an older version to replace pre-installed newer version. I had to choose to install older version to make the detector works.

I retried the colab notebook yesterday, this time if I still chose to replace preinstalled v1.5.0 by v.1.4.2, it will stuck at "building wheel for mmcv-full" for 20 mins and fail. If I chose not to replace preinstalled version and skip mmcv-full, the dependcie installing phase could be completed without error. But when I ran the detector, I got an error "KeyError: 'center'"

Please help.

KeyError                                  Traceback (most recent call last)
[<ipython-input-8-2cb6d21c10b9>](https://localhost:8080/#) in <module>()
     12 image = cv2.imread(input)
     13 
---> 14 preds = detector(image)

6 frames
[/content/anime-face-detector/anime_face_detector/detector.py](https://localhost:8080/#) in __call__(self, image_or_path, boxes)
    145                 boxes = [np.array([0, 0, w - 1, h - 1, 1])]
    146         box_list = [{'bbox': box} for box in boxes]
--> 147         return self._detect_landmarks(image, box_list)

[/content/anime-face-detector/anime_face_detector/detector.py](https://localhost:8080/#) in _detect_landmarks(self, image, boxes)
    101             format='xyxy',
    102             dataset_info=self.dataset_info,
--> 103             return_heatmap=False)
    104         return preds
    105 

[/usr/local/lib/python3.7/dist-packages/mmcv/utils/misc.py](https://localhost:8080/#) in new_func(*args, **kwargs)
    338 
    339             # apply converted arguments to the decorated method
--> 340             output = old_func(*args, **kwargs)
    341             return output
    342 

[/usr/local/lib/python3.7/dist-packages/mmpose/apis/inference.py](https://localhost:8080/#) in inference_top_down_pose_model(model, imgs_or_paths, person_results, bbox_thr, format, dataset, dataset_info, return_heatmap, outputs)
    385             dataset_info=dataset_info,
    386             return_heatmap=return_heatmap,
--> 387             use_multi_frames=use_multi_frames)
    388 
    389         if return_heatmap:

[/usr/local/lib/python3.7/dist-packages/mmpose/apis/inference.py](https://localhost:8080/#) in _inference_single_pose_model(model, imgs_or_paths, bboxes, dataset, dataset_info, return_heatmap, use_multi_frames)
    245                 data['image_file'] = imgs_or_paths
    246 
--> 247         data = test_pipeline(data)
    248         batch_data.append(data)
    249 

[/usr/local/lib/python3.7/dist-packages/mmpose/datasets/pipelines/shared_transform.py](https://localhost:8080/#) in __call__(self, data)
    105         """
    106         for t in self.transforms:
--> 107             data = t(data)
    108             if data is None:
    109                 return None

[/usr/local/lib/python3.7/dist-packages/mmpose/datasets/pipelines/top_down_transform.py](https://localhost:8080/#) in __call__(self, results)
    287         joints_3d = results['joints_3d']
    288         joints_3d_visible = results['joints_3d_visible']
--> 289         c = results['center']
    290         s = results['scale']
    291         r = results['rotation']

KeyError: 'center'

Thanks for your great work! May I ask which tool do you use to annotate the landmarks? I find the detector seems to perform not so well on the manga images. So I want to manually annotate some manga images. Besides, when you trained the landmarks detector, did you train the model from scratch or fine-tune on the pretrained mmpose model?

Thanks for your work! It’s very interesting!! May I ask you some questions? Did you manually annotate landmarks for the images generated by the TADNE model? And how many images does your training dataset include?

Thanks for sharing such a nice work! I was wondering if it is possible to implement anime face identification based on this detector. Do you have any plan on this? Will we have a good identification accuracy using this detector? Many thanks!

First of all, thank you for sharing your program.

Today I tried to run the program in GoogleColab and got the following error in the import anime_face_detector section. Do you know any solutions?

Thank you.

ImportError Traceback (most recent call last) in () 5 import numpy as np 6 ----> 7 import anime_face_detector

7 frames /usr/lib/python3.7/importlib/init.py in import_module(name, package) 125 break 126 level += 1 --> 127 return _bootstrap._gcd_import(name[level:], package, level) 128 129

ImportError: /usr/local/lib/python3.7/dist-packages/mmcv/_ext.cpython-37m-x86_64-linux-gnu.so: undefined symbol:_ZNK3c1010TensorImpl36is_contiguous_nondefault_policy_implENS_12MemoryFormatE

Hi, thanks for making a gradio demo for this on Huggingface https://huggingface.co/spaces/hysts/anime-face-detector, looks great with the new 3.0 design as well. Gradio has a event for the new Blocks API https://huggingface.co/Gradio-Blocks, it would be great if you can join to make a blocks version of this demo or another demo thanks!

awesome work!

especially face clustering very neat

this work reminds me of

How can Illustration be aligned and what can I do with these 2d landmark?

Scaling and rotating images and crop: FFHQ aligned code and webtoon result

Artstation-Artistic-face-HQ which counts as Illustration Use FFHQ aligned

and new FFHQ aligned https://arxiv.org/abs/2109.09378

but anime Illustration is not the same as real FFHQ, where perspective-related (pose) means destroying the centre, and local parts exaggeration destroying the global

[DO.1] directly k-mean dictionary (run a dataset) proximity aligned

Mention this analysis

[DO.2] because there are not many features can use, add continuous 2D spatial feature (pred), more point and even beyond

this need hack model (might proposed)

[DO.3] Should be used directly as a filter to assist with edge extraction (maximum reserve features)

guide VAE, SGF generation, or anime cross image Synthesis

if the purpose is not to train the generation model, probably use is to extend the dataset. if training to generate models, will greatly effect generated eye+chin centre aligned visual lines don't keeping real image features just polylines

Or need more key points in clustering, det box pts (easy [DO.4]), and beyond to the whole image

and thank for your reading this

result

by MPEG encoded that can't play properly (transcoded)

https://user-images.githubusercontent.com/26929386/141799892-0b496ada-66b4-4349-ab72-49aae2317ce4.mp4

comments

• not yet tested on gpu

• cleared all output

• didn't remove function detect , just copy the from demo_gradio.py

• polylines visualize function can be simplified

• polylines visualize function can be customize (color, thickness, groups)