Camel http4 download file using basic Https authentication
I am trying to download a file from an Https url that requires basic authentication. I am using HTTP4 I am trying to download from url - https://ebc.cybersource.com/ebc/DownloadReport/xxx.csv?authMethod=Basic&authUsername=scott&authPassword=tiger
After downloading the file, I need to save it to a folder. This is how my code looks like
from(xxx)
.to("http4://ebc.cybersource.com/ebc/DownloadReport/xxx.csv?authMethod=Basic&authUsername=scott&authPassword=tiger")
.to("file:target/messages/download");
Here's the error I am getting -
java.lang.UnsupportedOperationException: Cannot consume from http endpoint
at org.apache.camel.component.http4.HttpEndpoint.createConsumer(HttpEndpoint.java:120)
at org.apache.camel.impl.EventDrivenConsumerRoute.addServices(EventDrivenConsumerRoute.java:65)
at org.apache.camel.impl.DefaultRoute.onStartingServices(DefaultRoute.java:85)
at org.apache.camel.impl.RouteService.warmUp(RouteService.java:158)
at org.apache.camel.impl.DefaultCamelContext.doWarmUpRoutes(DefaultCamelContext.java:3090)
at org.apache.camel.impl.DefaultCamelContext.safelyStartRouteServices(DefaultCamelContext.java:3020)
at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRoutes(DefaultCamelContext.java:2797)
at org.apache.camel.impl.DefaultCamelContext.doStartCamel(DefaultCamelContext.java:2653)
at org.apache.camel.impl.DefaultCamelContext.access$000(DefaultCamelContext.java:167)
at org.apache.camel.impl.DefaultCamelContext$2.call(DefaultCamelContext.java:2467)
at org.apache.camel.impl.DefaultCamelContext$2.call(DefaultCamelContext.java:2463)
at org.apache.camel.impl.DefaultCamelContext.doWithDefinedClassLoader(DefaultCamelContext.java:2486)
at org.apache.camel.impl.DefaultCamelContext.doStart(DefaultCamelContext.java:2463)
at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:2432)
at com.dinesh.MainApp.main(MainApp.java:29)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
Now the documentation says that I need to use HttpContext,
public class HttpContextFactory {
private String httpHost = "localhost";
private String httpPort = 9001;
private BasicHttpContext httpContext = new BasicHttpContext();
private BasicAuthCache authCache = new BasicAuthCache();
private BasicScheme basicAuth = new BasicScheme();
public HttpContext getObject() {
authCache.put(new HttpHost(httpHost, httpPort), basicAuth);
httpContext.setAttribute(ClientContext.AUTH_CACHE, authCache);
return httpContext;
}
// getter and setter
}
I tried this but I am still getting errors.
UPDATE I am actually using quartz scheduler to run
from("quartz2://foo?cron=0/2+*+*+?+*+MON-FRI").multicast().stopOnException().to("direct:downLoadFile");
from("direct:downLoadFile")
.to("http4://ebc.cybersource.com/ebc/DownloadReport/2015/04/16/us_voip/us_voip.au.response.ss.csv?authMethod=Basic&authUserName=xxxx&authPassword=yyyy")
.log("File Downloaded");
When I run this I get the following error.
[ main] QuartzComponent INFO Starting scheduler.
[ main] QuartzScheduler INFO Scheduler DefaultQuartzScheduler-camel-1_$_NON_CLUSTERED started.
[ main] DefaultCamelContext INFO Total 2 routes, of which 2 is started.
[ main] DefaultCamelContext INFO Apache Camel 2.15.1 (CamelContext: camel-1) started in 0.938 seconds
[artzScheduler-camel-1_Worker-1] RetryExec INFO I/O exception (java.net.SocketException) caught when processing request to {}->http://ebc.cybersource.com:80: Connection reset
[artzScheduler-camel-1_Worker-1] RetryExec INFO Retrying request to {}->http://ebc.cybersource.com:80
[artzScheduler-camel-1_Worker-1] RetryExec INFO I/O exception (java.net.SocketException) caught when processing request to {}->http://ebc.cybersource.com:80: Connection reset
[artzScheduler-camel-1_Worker-1] RetryExec INFO Retrying request to {}->http://ebc.cybersource.com:80
[artzScheduler-camel-1_Worker-1] RetryExec INFO I/O exception (java.net.SocketException) caught when processing request to {}->http://ebc.cybersource.com:80: Connection reset
[artzScheduler-camel-1_Worker-1] RetryExec INFO Retrying request to {}->http://ebc.cybersource.com:80
[artzScheduler-camel-1_Worker-1] DefaultErrorHandler ERROR Failed delivery for (MessageId: ID-CN-AHONNAVA-LP-49362-1430485742705-0-1 on ExchangeId: ID-CN-AHONNAVA-LP-49362-1430485742705-0-3). Exhausted after delivery attempt: 1 caught: java.net.SocketException: Connection reset
Message History
---------------------------------------------------------------------------------------------------------------------------------------
RouteId ProcessorId Processor Elapsed (ms)
[route1 ] [route1 ] [quartz2://foo?cron=0%2F2+*+*+%3F+*+MON-FRI ] [ 523]
[route1 ] [multicast1 ] [multicast ] [ 520]
[route1 ] [to1 ] [direct:downLoadFile ] [ 513]
[route2 ] [to2 ] [http4://ebc.cybersource.com/ebc:443/DownloadReport/2015/04/16/us_voip/us_voip.] [ 510]
Exchange
---------------------------------------------------------------------------------------------------------------------------------------
Exchange[
Id ID-CN-AHONNAVA-LP-49362-1430485742705-0-3
ExchangePattern InOnly
Headers {breadcrumbId=ID-CN-AHONNAVA-LP-49362-1430485742705-0-1, calendar=null, CamelRedelivered=false, CamelRedeliveryCounter=0, fireTime=Fri May 01 09:09:04 EDT 2015, jobDetail=JobDetail 'Camel_camel-1.foo': jobClass: 'org.apache.camel.component.quartz2.CamelJob concurrentExectionDisallowed: false persistJobDataAfterExecution: false isDurable: false requestsRecovers: false, jobInstance=org.apache.camel.component.quartz2.CamelJob@6e61a2e3, jobRunTime=-1, mergedJobDataMap=org.quartz.JobDataMap@2a207ff6, nextFireTime=Fri May 01 09:09:06 EDT 2015, previousFireTime=null, refireCount=0, result=null, scheduledFireTime=Fri May 01 09:09:04 EDT 2015, scheduler=org.quartz.impl.StdScheduler@1e79d438, trigger=Trigger 'Camel_camel-1.foo': triggerClass: 'org.quartz.impl.triggers.CronTriggerImpl calendar: 'null' misfireInstruction: 1 nextFireTime: Fri May 01 09:09:06 EDT 2015, triggerGroup=Camel_camel-1, triggerName=foo}
BodyType null
Body [Body is null]
]
Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:196)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:136)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:152)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:270)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:161)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:153)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:254)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
at org.apache.camel.component.http4.HttpProducer.executeMethod(HttpProducer.java:258)
at org.apache.camel.component.http4.HttpProducer.process(HttpProducer.java:158)
at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:129)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:51)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:129)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:590)
at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:518)
at org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:227)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:44)
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:109)
at org.apache.camel.processor.loadbalancer.LoadBalancerSupport.process(LoadBalancerSupport.java:87)
at org.apache.camel.component.quartz2.CamelJob.execute(CamelJob.java:56)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
[artzScheduler-camel-1_Worker-1] CamelJob ERROR Error processing exchange. Exchange[Message: [Body is null]]. Caused by: [org.quartz.JobExecutionException - org.apache.camel.CamelExchangeException: Sequential processing failed for number 0. Exchange[Message: [Body is null]]. Caused by: [java.net.SocketException - Connection reset]]
[artzScheduler-camel-1_Worker-1] JobRunShell INFO Job Camel_camel-1.foo threw a JobExecutionException:
org.quartz.JobExecutionException: org.apache.camel.CamelExchangeException: Sequential processing failed for number 0. Exchange[Message: [Body is null]]. Caused by: [java.net.SocketException - Connection reset] [See nested exception: org.apache.camel.CamelExchangeException: Sequential processing failed for number 0. Exchange[Message: [Body is null]]. Caused by: [java.net.SocketException - Connection reset]]
at org.apache.camel.component.quartz2.CamelJob.execute(CamelJob.java:59)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
FINAL UPDATE: SOLVED Thanks to @Claus Ibsen who provided the correct component. All I needed was to use https4 instead of http. Note that this information is missing from the camel documentation. Here's the final code that downloads the file from the website and stores it in the target / download folder.
import org.apache.camel.builder.RouteBuilder;
import org.apache.commons.codec.binary.Base64;
/**
* Created by darora on 4/30/2015.
*/
public class SchedulerRoute extends RouteBuilder {
//TODO Move this to properties file
String userCredentials ="userName:Password";
String basicAuth = "Basic " + new String(new Base64().encode(userCredentials.getBytes()));
@Override
public void configure() throws Exception {
//from("timer://foo?fixedRate=true&period=6000").to("bean:logBean?method=sayHello");
//from("scheduler://foo?delay=500").to("bean:logBean?method=sayHello");
from("quartz2://foo?cron=0/2+*+*+?+*+MON-FRI").multicast().stopOnException().to("direct:downLoadFile","direct:myBean");
from("direct:myBean").log("Scheduler Started one more time .....")
.to("bean:downLoadFileBean?method=updatedatabase");
from("direct:downLoadFile").log("Scheduler Started one more time .....")
.setHeader("Authorization",constant(basicAuth))
.to("https4://ebc.cybersource.com/ebc/DownloadReport/2015/04/16/us_voip/us_voip.au.response.ss.csv")
.to("file:target/download")
.log("file is downloaded ..........");
}
}
+3
source to share
1 answer
You cannot start with http4 as it tells you you need a timer or something to invoke the route
from("timer:foo?period=5000")
.to("http4://ebc.cybersource.com/ebc/DownloadReport/xxx.csv?authMethod=Basic&authUsername=scott&authPassword=tiger")
.to("file:target/messages/download");
In this example, the timer starts every 5 seconds.
+6
source to share