当前位置: X-MOL 学术arXiv.cs.CL › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Discoverability in Satellite Imagery: A Good Sentence is Worth a Thousand Pictures
arXiv - CS - Computation and Language Pub Date : 2020-01-03 , DOI: arxiv-2001.05839
David Noever, Wes Regian, Matt Ciolino, Josh Kalin, Dom Hambrick, Kaye Blankenship

Small satellite constellations provide daily global coverage of the earth's landmass, but image enrichment relies on automating key tasks like change detection or feature searches. For example, to extract text annotations from raw pixels requires two dependent machine learning models, one to analyze the overhead image and the other to generate a descriptive caption. We evaluate seven models on the previously largest benchmark for satellite image captions. We extend the labeled image samples five-fold, then augment, correct and prune the vocabulary to approach a rough min-max (minimum word, maximum description). This outcome compares favorably to previous work with large pre-trained image models but offers a hundred-fold reduction in model size without sacrificing overall accuracy (when measured with log entropy loss). These smaller models provide new deployment opportunities, particularly when pushed to edge processors, on-board satellites, or distributed ground stations. To quantify a caption's descriptiveness, we introduce a novel multi-class confusion or error matrix to score both human-labeled test data and never-labeled images that include bounding box detection but lack full sentence captions. This work suggests future captioning strategies, particularly ones that can enrich the class coverage beyond land use applications and that lessen color-centered and adjacency adjectives ("green", "near", "between", etc.). Many modern language transformers present novel and exploitable models with world knowledge gleaned from training from their vast online corpus. One interesting, but easy example might learn the word association between wind and waves, thus enriching a beach scene with more than just color descriptions that otherwise might be accessed from raw pixels without text annotation.

中文翻译:

卫星图像的可发现性:一句好话值一千张图片

小型卫星星座提供地球陆地的每日全球覆盖,但图像丰富依赖于自动化关键任务,如变化检测或特征搜索。例如,从原始像素中提取文本注释需要两个相关的机器学习模型,一个用于分析头顶图像,另一个用于生成描述性标题。我们在之前最大的卫星图像字幕基准上评估了七个模型。我们将标记的图像样本扩展五倍,然后扩充、校正和修剪词汇表以接近粗略的 min-max(最小单词,最大描述)。这一结果与之前使用大型预训练图像模型的工作相比具有优势,但在不牺牲整体准确性的情况下将模型大小减小了 100 倍(当使用对数熵损失测量时)。这些较小的模型提供了新的部署机会,特别是在推送到边缘处理器、机载卫星或分布式地面站时。为了量化标题的描述性,我们引入了一种新颖的多类混淆或错误矩阵来对人工标记的测试数据和包括边界框检测但缺少完整句子标题的未标记图像进行评分。这项工作提出了未来的字幕策略,特别是那些可以丰富土地使用应用以外的类别覆盖范围并减少以颜色为中心和相邻的形容词(“绿色”、“附近”、“之间”等)的策略。许多现代语言转换器提供了新颖且可利用的模型,其中包含从其庞大的在线语料库中收集的世界知识。一个有趣的,
更新日期:2020-01-17
down
wechat
bug