Rather than the time slice idea, allow us to schedule pipelines as jobs, the same way I would schedule an agent job to run SSIS packages. Setting availability for datasets is a very awkward way to go about this. A scheduler would be 10 times easier and more intuitive.

Also allow users to "run" a pipeline on demand, this would make testing a lot easier.

The JSON editor is OK but is still a barrier to entry. A WYSIWIG UI based on SSIS/Machine Learning Studio would really make this easier to use.

Power Query is awesome! It would be a great feature to be able to output its result into either a SQL database or Azure (Storage or SQL).

Activity that copies and then deletes.

It'd make it much easier to adopt Data Factory if it was possible to add Azure Functions activities into a Pipeline.

You can already store a blob and make an Azure Function to trigger based on that, but having the functions directly in the pipeline source would make the Data Factory management easier. Not to mention the clarity it'd give about the Data Factory functionality.

At the moment you can only use * and ? in the file filter. It would be very helpful if you could use the partitionedBy section which you can use for the folderpath in the filefilter or the filename as well.

This would allow scenarios where you need files like myName-2015-07-01.txt where the slice date and time is part of the filename.

Now that we have Azure functions available that can help make batch data processing more real-time, it would be great to be able to programmatically invoke the workflow component of ADF, for immediate execution of the pipeline.

Source and sink.

We have around 100 datasets and 10 different pipelines. This will grow to 2000 datasets and 150 pipelines in future based on business functionality, data categorization and dependency. We already see a problem in managing it, as we are not able to arrange them in a folder(collapse) and diagram for it becomes difficult to explain. If we introduce a functionality to manage all datasets/pipeline related to one area in a specific folder and also have diagram feature specific to that folder level, it will simplify it a lot. See Before and After attachment for more clarity.

Let's enable the hottest Big Data technology of the day as a data hub. This enables an in memory ELT capability to the ADF family.

Can it consume or push data to Azure app service API? Supporting Swagger API.

There are many cloud applications that expose data via a SOAP or REST api. Customers should be able to configure generic REST and SOAP data sources for use in Azure Data Factory. Other ELT and ETL tools such as Dell Boomi, Informatica, SSIS and Talend have this functionality.

O365 OneRM team is solving the same problems that you are, and has a mature platform that does many of the things that Azure DF will need to do. Talk to Zach, Naveen, and Karthik in building 2. Also talk to Pramod. It'll accelerate you in terms of battle-hardened user needs and things pipeline automation has needed to do in the field. You'll want to get a copy of the bits/code.

the web and odata connectors need to add support for OAuth ASAP. Most other Microsoft services (Office 365, PWA, CRM, etc, etc, etc) along with many other industry API's require the use of OAuth. Not having this closes the door to lots of integration scenarios.

As of now, portal shows status as In Progress or Running, but doesn't provide in-depth detail or logs of pipeline activity that is In Progress. For example, I started a copy activity from Azure table storage to Azure blob storage. It has been running for last 15 hours, but it doesn't give any idea around, how many rows or data volume that has been copied already. For any activity, the status can be provided every few mins.

There should be a option to clear old errors.
When there is no pipeline that produces or consumes a data slice, and this slice has errors the counter still shows "current" errors, and this is not the case. I would like to remove these unused slices and their errors.

source and sink.

GUI interface similar to Azure ML UX experience