Data Engineer Storytelling
About PageshiftPageshift is a Research Lab committed to pushing the frontier of AI storytelling and creativity. We are envisioning a world in which most entertainment is personalized and AI-generated. Our goal is to build the underlying story engine that powers it all. To do this, we are not afraid to explore new ways and create novel categories of model capability.About the roleYou are expected to process books and other written stories into structured datasets, with a strong focus on quality. You will build pipelines that extract complex narrative aspects and you will validate results through systematic sampling and manual review. You are also expected to build pipelines that evaluate our model generations for the correct use of storytelling concepts.What we're looking for:- Passion for storytelling
- Cares about data and data quality
- Willing to manually read a bunch of data
- Understanding of storytelling, narrative flows, world building and character development
- Basic understanding of LLM prompting, or willingness to learn
- Basic understanding of Python and data processing, or willingness to learnNice to have:- Have built some prior datasets
- Have done some theoretical-based analysis of stories
- Have a deep understanding of the limitations of LLMs when it comes to the understanding of stories and narrativesYour responsibilities:- Develop and validate LLM-driven data processing pipelines for books and other forms of written stories
- Build LLM-based validation pipelines for aspects of books and stories
