diff --git a/README.md b/README.md index 4c7e374c74f6ec0424cbc22034cb8c62c5266ef6..2a01455971582aa741f558dfc05e4dd3255ffc73 100644 --- a/README.md +++ b/README.md @@ -1,30 +1,23 @@ ## Evaluating and Extending Unsupervised VideoSummarization Methods -1. Evaluation unsupervised methods with different metric under same configuration. -2. Investigating effect of extracted features in unsupervised methods and extend it to perform better than baseline work. -3. Identify gaps form last step and try to fill by porposed solution by extending or modifying an existing model. +1. Reproduce unsupervised method CSNet and RL-method SUM-Ind. +2. Evaluating SUM-GAN-AAE, SUM-GAN-sl, CSNet, and SUM-Ind using F1-score and rank correlation coefficients. +2. Extending CSNet with features variation by applying fusion techniques. -**Abstract** - -This study validates the recent work in unsupervised video summarization and extends the experiments with feature variation. -In our understanding, deep learning based approaches select frame-candidates for a video summary. -These frames are drawn with a probability that can be used to calculate the scaled importance for comparison with the basic truth as an evaluation. -However, most works in the literature focus on network architecture and follow the same feature extraction technique using the resulting deep features from one pretrained model. -At this point a notable gap of the existing approaches is feature variation, which this study explores. -Building on that, model variables like network architecture, optimizer, and activation functions can have an impact on the performance combined with different feature selection techniques that haven’t beenexplored yet.Further, the Evaluation of existing models is conducted using one metric,which may be not representative in video summarization task, since it could ignore key-frames in favor of other frames and still have a relatively high value. This work performs feature extraction using multiple pretrained neural models, and then measures the impact of them on current state-of-the-art works.Then,it evaluates the state-of-the-art works using different evaluation metrics. -Eventually, it aims to find an unsupervised video summarization method, in order to fill the gaps, and leverage the existing works. - -**Important Wiki Pages:** - -* [Notes on SUM-GAN-AAE (Apostolidis et al. 2020)](https://gitlab.uni-hannover.de/hussainkanafani/unsupervised-video-summarization/-/wikis/SUM-GAN-AAE-(Apostolidis-et-al.-2020)) - -* [References and important findings](https://gitlab.uni-hannover.de/hussainkanafani/unsupervised-video-summarization/-/wikis/Findings) - -* [Reproduce (Zhou et al. 2018)](https://gitlab.uni-hannover.de/hussainkanafani/unsupervised-video-summarization/-/wikis/Reproduce-(Zhou-et-al.-2018)%E2%80%8B) - -* [Conda cheat sheet](https://gitlab.uni-hannover.de/hussainkanafani/unsupervised-video-summarization/-/wikis/Conda-cheat-sheet) +# Project Structure +``` +Directory: +- /data +- /csnet (implementation of csnet method) +- /src/evaluation (evaluation using F1-score and rank correlations coefficients) +- /src/visualization +- /sum-ind (implementation of SUM-Ind method) +- /CSNET-places365-early-fusion (Fusion variation based on CSNet) +- /CSNET-places365-late-fusion (Fusion variation based on CSNet) +- /CSNET-places365-intermediate-fusion (Fusion variation based on CSNet) +``` # Datasets Structured h5 files with the video features and annotations of the SumMe and TVSum datasets are available within the "data" folder. The GoogleNet features of the video frames were extracted by [Ke Zhang](https://github.com/kezhang-cs) and [Wei-Lun Chao] and the h5 files were obtained from [Kaiyang Zhou](https://github.com/KaiyangZhou/pytorch-vsumm-reinforce). @@ -79,6 +72,18 @@ python main.py -d datasets/eccv16_dataset_summe_google_pool5.h5 -s datasets/summ ``` +**Important Wiki Pages:** + +* [Notes on SUM-GAN-AAE (Apostolidis et al. 2020)](https://gitlab.uni-hannover.de/hussainkanafani/unsupervised-video-summarization/-/wikis/SUM-GAN-AAE-(Apostolidis-et-al.-2020)) + +* [References and important findings](https://gitlab.uni-hannover.de/hussainkanafani/unsupervised-video-summarization/-/wikis/Findings) + +* [Reproduce (Zhou et al. 2018)](https://gitlab.uni-hannover.de/hussainkanafani/unsupervised-video-summarization/-/wikis/Reproduce-(Zhou-et-al.-2018)%E2%80%8B) + +* [Conda cheat sheet](https://gitlab.uni-hannover.de/hussainkanafani/unsupervised-video-summarization/-/wikis/Conda-cheat-sheet) + + + ### Citations ``` @article{zhou2017reinforcevsumm,