背景
在產品環境上通過 http 的方式訪問 aws s3 是不安全的,需要使用aws sdk 提供的接口來訪問
技術實現
項目中使用的是java
1. 在gradel 中引用對應的aws 包
implementation ‘software.amazon.awssdk:s3:2.20.80’ // aws sdk
implementation ‘software.amazon.awssdk:sts:2.20.0’
2. 在gradel 中引用對應的aws 包
需要使用S3 Client 來向aws s3 發起請求,為了避免重復創建Client 冗余對象,創建工廠類來管理對應的 S3 Client 對象。擴展性也大幅提高。
此外,由于每一個Client 對象是一個單一的Region,需要不同的aws cluster_region 。
public class S3ClientFactory {private static final Map<String, S3Client> clients = new ConcurrentHashMap<>();public static S3Client getClient(Region region) {return clients.computeIfAbsent(region.toString(), r -> S3Client.builder().region(Region.of(r)).build());}public static String getCsvContent(String bucketName, String objectKey, String configKey) {// Create S3 ClientsS3Client s3Stg = S3ClientFactory.getClient(Region.US_WEST_2); // stageS3Client s3ProdNa = S3ClientFactory.getClient(Region.US_EAST_1); // prod NA// Select the appropriate S3 client based on configKeyS3Client s3 = selectClient(configKey, s3Stg, s3ProdNa);// Prepare the GetObject requestGetObjectRequest getObjectRequest = GetObjectRequest.builder().bucket(bucketName).key(objectKey).build();// Fetch the object and read the content into a Stringtry (ResponseInputStream<?> response = s3.getObject(getObjectRequest);BufferedReader reader = new BufferedReader(new InputStreamReader(response))) {// Collect all lines into a single stringreturn reader.lines().collect(Collectors.joining("\n"));} catch (S3Exception e) {throw new CustomException("Failed to get AWS CSV by bucket and objectKey: " + e.awsErrorDetails().errorMessage(), e.getMessage());} catch (IOException e) {throw new CustomException("Error processing CSV content from AWS S3: " + e.getMessage(), e.getMessage());}}private static S3Client selectClient(String configKey, S3Client s3Stg, S3Client s3ProdNa) {if (configKey.contains("prod")) {return s3ProdNa; // Assuming s3ProdNa is used for NA and SA} else {return s3Stg; // Default to stage}}
}
展望
搞清楚 aws sdk 權限,access_api key,訪問通信原理