Application of a Multimodal Transformer to the Prediction of the Yield of Saturated Hydrocarbon Compounds from Heavy Crude Oil in the Presence of Catalysts

Heavy oil fields are a promising energy source in the future due to the depletion of natural sources of light oil. However, extraction, transportation and refining of heavy oil is significantly more complicated than light oil - difficulties arise at almost all technological stages. One of such stages is laboratory analytics of heavy oil and selection of the most optimal catalyst for extraction of required fractions from crude oil sample. Different catalysts are actively used in petrochemical laboratories, but special attention is paid to those of them, the basis of which is metal. In this study, catalysts based on six different metals namely zinc, nickel, copper, manganese, lead and sodium were analyzed. In order to analyze the yield ratios of the required components from the crude heavy oil composition, it is necessary to test different types of catalysts sequentially on a base sample. The yield of different hydrocarbons on a small volume of oil can be reliably estimated by chromatographic study, which takes about 68 minutes for both the base oil sample and the different catalysts. Since testing 6 different catalysts would require almost 7 hours of chromatographic analysis, a rational solution would be to apply data mining techniques to this task. A multimodal transformer model was proposed to solve this problem. It takes as input two modalities: a chromatogram of a sample of pure crude oil presented as graphical data and accompanying tabular data, which are also generated by the chromatograph and consist of text and numbers. At the output, the model produces predictive tabular data that formalize the redistributed group composition of the oil and describe both the names of the newly produced hydrocarbons and their two qualitative characteristics: time and yield area. Obtaining the prediction makes it possible to significantly reduce the time, hardware and human resources required to select the right type of catalyst in petrochemical laboratories. In the process of the study, it was found that training of the intellectual model on the data of one field allows to perform further similar forecast with acceptable accuracy for the data of another heavy oil field. The magnitude of the prediction error of the intelligent model satisfies the requirements set by the petrochemical laboratory for practical application of the multimodal transformer.

1. Шадрина А. В., Крец В. Г. Основы нефтегазового дела. М., Нац. Открытый Ун-т «ИНТУИТ», 2016. 214 с. / Shadrina, A.V., Krets, V.G. Fundamentals of Oil and Gas Engineering. Moscow, National Open University of INTUIT, 2016, 214 p. (in Russian).

2. Басарыгин Ю. М., Будников В. Ф., Булатов А. И. Исследование факторов и реализация мер долговременной эксплуатации нефтяных и газовых скважин. М., Просвещение-Юг, 2004, 242 с. ISBN 5-93491-054-X. / Basarygin Y. M., Budnikov V. F., Bulatov A. I. Research of factors and implementation of measures for long-term operation of oil and gas wells. Moscow, Prosveshchenie-Yug, 2004, 242 p. (in Russian). ISBN 5-93491-054-X.

3. Шевченко Д. В., Васильева Л. Х. Математическое моделирование вытеснения тяжелых нефтей горячей водой в тонком пласте. Казань, Издательство "Познание", 2013, 60 с. / Shevchenko D. V., Vasilieva L. H. Mathematical modelling of displacement of heavy oils by hot water in a thin reservoir. Kazan, Poznanie Publishing House, 2013, 60 p. (in Russian).

4. Хуснутдинов И. Ш., Копылов А. Ю., Гончарова И. Н. Разработка и совершенствование сольвентных технологий переработки тяжелого органического сырья. Казань, КГТУ, 2009, 265 с. ISBN 978-5-7882-0745-2 / Khusnutdinov I. Sh., Kopylov A. Yu., Goncharova I. N. Development and improvement of solvent technologies for processing of heavy organic raw materials. Kazan, KSTU, 2009, 265 p. (in Russian). ISBN 978-5-7882-0745-2.

5. Хадавимогаддам Ф., Мищенко И. Т., Мостаджеран М. Применение методов искусственного интеллекта в прогнозировании основных свойств нефти. Газовая промышленность, вып. 12(794), 2019, стр. 28-32. / Hadavimoghaddam F., Mishchenko I. T., Mostajeran M. Application of artificial intelligence methods in predicting basic oil properties. Gas Industry, vol. 12(794), 2019, pp. 28-32 (in Russian).

6. Азиева Р. Х. Искусственный интеллект в добыче нефти и газа: возможности и сценарный прогноз. Проблемы экономики и управления нефтегазовым комплексом, вып. 3(207), 2022, стр. 38-46. DOI: 10.33285/1999-6942-2022-3(207)-38-46. / Azieva R. H. Artificial intelligence in oil and gas production: opportunities and scenario forecast. Problems of economics and management of oil and gas complex, vol. 3(207), 2022, pp. 38-46 (in Russian). DOI: 10.33285/1999-6942-2022-3(207)-38-46.

7. Байбаров Д. А. Оценка продуктивности и экономической эффективности технологий искусственного интеллекта для автоматизации процессов разведки и добычи нефти и газа. XXI век: итоги прошлого и проблемы настоящего плюс, том 10, вып. 3, 2021, стр. 100-105. DOI: 10.46548/21vek-2021-1055-0019 / Baybarov D. A. Assessment of productivity and economic efficiency of artificial intelligence technologies for automation of oil and gas exploration and production processes. XXI century: Resumes of the Past and Challenges of the Present plus, vol. 10, issue 3, 2021, pp. 100-105 (in Russian). DOI: 10.46548/21vek-2021-1055-0019

8. Ломакин Н. И., Дженифер О. Ч., Голодова О. А., Сычева А. В., Кабина В. В. AI-система "Персептрон" для прогноза финансового результата деятельности предприятия нефтяной отрасли РФ. Фундаментальные исследования, вып. 12-1, 2019, стр. 98-103. DOI: 10.17513/fr.42629 / Lomakin N. I., Jennifer O. C., Golodova O. A., Sycheva A. V., Kabina V. V. AI-system "Perseptron" for forecasting the financial result of the enterprise of the oil industry of the Russian Federation. Fundamental Research, vol. 12-1, 2019, pp. 98-103 (in Russian). DOI: 10.17513/fr.42629

9. Овсеенко Г. А., Козелков О. В., Кашаев Р. С. Использование нейронных сетей в мехатронном устройстве представительного отбора и анализа проб. Приборостроение и автоматизированный электропривод в топливно-энергетическом комплексе и жилищно-коммунальном хозяйстве. Материалы VII Национальной научно-практической конференции, 2022, стр. 92-96. / Ovseenko G. A., Kozelkov O. V., Kashaev R. S. Use of neural networks in mechatronic device of representative sampling and analysis. Instrumentation and automated electric drive in fuel and energy complex and housing and communal services. Proceedings of VII National Scientific and Practical Conference, 2022, pp. 92-96 (in Russian).

10. Vaswani A., Shazeer N. et al. Attention is all you need. In Proc. of the 31st Conference on Neural Information Processing Systems (NIPS), 2017, 11 p.

11. Dosovitskiy A., Beyer L. et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In Proc. of the International Conference on Learning Representations (ICLR), 2021, 21 p. DOI: 10.48550/arXiv.2010.11929

12. Arik S. Ö., Pfister T. TabNet: Attentive Interpretable Tabular Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 35(8), 2020, pp. 6679-6687. DOI: 10.1609/aaai.v35i8.16826