In Spring Boot uploading files to S3 can be implemented using the REST API POST/PUT method very easily. This can be implemented using Spring WebMVC and was s3 SDK. I this example I used the following dependencies using Gradle.
Gradle dependencies
// https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-web
api 'org.springframework.boot:spring-boot-starter-web:2.4.1'
// This dependency is exported to consumers, that is to say found on their compile classpath.
api 'org.apache.commons:commons-math3:3.6.1'
// https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-s3
api 'com.amazonaws:aws-java-sdk-s3:1.11.924'
// https://mvnrepository.com/artifact/org.apache.tika/tika-core
api 'org.apache.tika:tika-core:1.25'
// https://mvnrepository.com/artifact/commons-io/commons-io
api 'commons-io:commons-io:2.8.0'
// https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-hateoas
api 'org.springframework.boot:spring-boot-starter-hateoas:2.4.1'
Spring boot config for async upload file to AWS S3
We need to define one bean for AmazonS3 and another for ThreadPoolTaskExecutor. See the below code for the definitions.
@Configuration
public class ServiceConfig {
@Value("${AWSAccessKeyId}")
private String accessKeyId;
// Secret access key will be read from the application.properties file during the application intialization.
@Value("${AWSSecretKey}")
private String secretAccessKey;
private final String region="ap-south-1";
@Bean
public AmazonS3 getAmazonS3Cient() {
final BasicAWSCredentials basicAWSCredentials = new BasicAWSCredentials(this.accessKeyId, this.secretAccessKey);
// Get AmazonS3 client and return the s3Client object.
return AmazonS3ClientBuilder
.standard()
.withRegion(Regions.fromName(this.region))
.withCredentials(new AWSStaticCredentialsProvider(basicAWSCredentials))
.build();
}
@Bean(name="taskExecutor")
public Executor taskExecutor() {
final ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(10);
executor.setMaxPoolSize(10);
executor.setQueueCapacity(500);
executor.setThreadNamePrefix("S3Uploader-");
executor.initialize();
return executor;
}
}
Service for Async Upload to AWS S3
Below code is used to upload file to S3
@Service
public class S3UploadServiceImpl implements UploadService{
@Autowired
private AmazonS3 amazonS3;
private final String bucket= "test.bimbim.in";
//Change this with db
private final ConcurrentMap<String, FileMeta> uploadTrackMap = new ConcurrentHashMap<String, FileMeta>();
@Override
public FileMeta get(final String id) {
return this.uploadTrackMap.get(id);
}
@Async("taskExecutor")
@Override
public void process(final byte[] bs, final FileMeta meta){
this.upload(bs, meta);
}
@Override
public void upload(final byte[] bs, final FileMeta meta) {
this.uploadTrackMap.put(meta.getId(), meta);
final String fileName = meta.getId()+meta.getFileName();
final InputStream targetStream = new ByteArrayInputStream(bs);
final PutObjectRequest putObjectRequest = new PutObjectRequest(this.bucket,fileName, targetStream, new ObjectMetadata());
this.amazonS3.putObject(putObjectRequest);
final String url = this.amazonS3.getUrl(this.bucket, fileName).toString();
meta.setS3Path(url);
meta.setUploaded(true);
}
}
Controller to upload File
Below controller code used to received file buffered and involve service upload method to upload file aync.
@RestController
public class UploadController {
@Autowired
private UploadService uploadService;
@RequestMapping(value = "/api/upload/{id}/", method = RequestMethod.GET)
public ResponseEntity<Object> get(@PathVariable("id") final String id){
final FileMeta meta = this.uploadService.get(id);
if(meta==null) {
return ResponseEntity.notFound().build();
}else if(meta.isUploaded()) {
return ResponseEntity.ok(meta.add(linkTo(methodOn(UploadController.class).get(id)).withSelfRel()));
}else {
final ResourceStatus status = new ResourceStatus();
status.setStatus("In Progress");
return ResponseEntity.ok(status.add(linkTo(methodOn(UploadController.class).get(id)).withRel("delete")));
}
}
@RequestMapping(value = "/api/upload/", method = RequestMethod.POST)
public ResponseEntity<Object> upload(@RequestParam final MultipartFile file) throws IOException {
if(file != null && !file.isEmpty()) {
final FileMeta meta = new FileMeta();
meta.setFileName(file.getOriginalFilename());
meta.setLength(file.getSize());
try {
meta.setType( C4CFileUtils.getMediaType(file));
} catch (final IOException e) {
//No able to parse
//Do exception handling
}
meta.setUploadedAt(Calendar.getInstance());
final byte buffer[] = new byte[(int)file.getSize()];
IOUtils.readFully(file.getInputStream(), buffer);
this.uploadService.process(buffer, meta);
return ResponseEntity.accepted().header("location", "/api/upload/"+meta.getId()+"/").build();
}else {
return new ResponseEntity<Object>(HttpStatus.BAD_REQUEST);
}
}
}
You can find the complete source on the GIT hub https://github.com/sheelprabhakar/spring-boot-examples/tree/main/spring-async-uploader-s3