首先要Hadoop版本支持,其次要配置hdfs-site.xml文件:
<property>
<name>dfs.support.append</name>
<value>true</value>
</property>
话不多说,代码如下:
/**
* Append text lines to a HDFS file.
*
* @param conf The Hadoop {@link Configuration} object.
* @param path The output HDFS file path.
* @param data The input data to write.
* @return true for success, false otherwise.
*/
public static boolean writeLinesToHdfs(Configuration conf, Path path, List<String> data) {
FSDataOutputStream outputStream = null;
FileSystem fs = null;
try {
fs = FileSystem.get(conf);
outputStream = fs.append(path);
for (String line : data) {
outputStream.write(line.getBytes(StandardCharsets.UTF_8));
outputStream.write("\n".getBytes(StandardCharsets.UTF_8));
}
} catch (IOException e) {
return false;
} finally {
IOUtils.closeQuietly(fs);
IOUtils.closeQuietly(outputStream);
}
return true;
}
文章来源:https://www.codelast.com/
测试结果:调用此方法,文件确实能以追加的形式写入。