script-to-movie: a computational framework for story movie composition

ISCAS OpenIR

	script-to-movie: a computational framework for story movie composition
	Liang Chao; Xu Changsheng; Cheng Jian; Min Weiqing; Lu Hanqing
	2013
发表期刊	IEEE Transactions on Multimedia
ISSN	1520-9210
卷号	15 期号:2 页码:401-414
摘要	Traditional movie production has always been a highly professional work that needs team collaboration, advanced devices and techniques, and vast time and money investment. These high threshold requirements not only prevent mass amateur enthusiasts entering this field, but also hinder professionals quickly previewing their conceived story plots. In this paper, we raise a novel application, named script-to-movie (S2M) composition, to automatically produce new movies from existing videos in accordance with user created script. Our motivation is to liberate producers from complex filming and editing operations, thereby people's story idea can be instantly converted into the vivid movie video. To support the novel 'What You Dream Is What You See' (WYDIWYS) production mode, we first propose a hierarchical alignment method to automatically construct a video material database with detailed semantic description. Considering diverse story plots in user designed script, the database contains abundant video materials about different characters appearing in various time and places conditions. On this basis, the S2M composition is formulated as a constrained optimization problem, where semantic story plot and syntactic visual content are synthetically considered to identify a group of optimal video segments to narrate the user designed script story. Both quantitative and qualitative experimental results are reported to illustrate the effectiveness of the proposed S2M application. © 1999-2012 IEEE.; Traditional movie production has always been a highly professional work that needs team collaboration, advanced devices and techniques, and vast time and money investment. These high threshold requirements not only prevent mass amateur enthusiasts entering this field, but also hinder professionals quickly previewing their conceived story plots. In this paper, we raise a novel application, named script-to-movie (S2M) composition, to automatically produce new movies from existing videos in accordance with user created script. Our motivation is to liberate producers from complex filming and editing operations, thereby people's story idea can be instantly converted into the vivid movie video. To support the novel 'What You Dream Is What You See' (WYDIWYS) production mode, we first propose a hierarchical alignment method to automatically construct a video material database with detailed semantic description. Considering diverse story plots in user designed script, the database contains abundant video materials about different characters appearing in various time and places conditions. On this basis, the S2M composition is formulated as a constrained optimization problem, where semantic story plot and syntactic visual content are synthetically considered to identify a group of optimal video segments to narrate the user designed script story. Both quantitative and qualitative experimental results are reported to illustrate the effectiveness of the proposed S2M application. © 1999-2012 IEEE.
收录类别	EI
关键词	Constrained Optimization Semantics
部门归属	(1) National Lab of Pattern Recognition Institute of Automation Chinese Academy of Sciences Beijing 100190 China; (2) National Engineering Research Center on Multimedia Software School of Computer Science Wuhan University Wuhan 430079 China; (3) China-Singapore Institute of Digital Media Singapore 119615 Singapore
语种	英语
内容类型	期刊论文
URI标识	http://ir.iscas.ac.cn/handle/311060/15224
专题	中国科学院软件研究所
推荐引用方式 GB/T 7714	Liang Chao,Xu Changsheng,Cheng Jian,et al. script-to-movie: a computational framework for story movie composition[J]. IEEE Transactions on Multimedia,2013,15(2):401-414.
APA	Liang Chao,Xu Changsheng,Cheng Jian,Min Weiqing,&Lu Hanqing.(2013).script-to-movie: a computational framework for story movie composition.IEEE Transactions on Multimedia,15(2),401-414.
MLA	Liang Chao,et al."script-to-movie: a computational framework for story movie composition".IEEE Transactions on Multimedia 15.2(2013):401-414.