Is there a limit to the number of inputs on a Magic ETL dataflow?

MattGo
MattGo Member
in Archive

Beginning to create a dataflow in ETL that will require 100+ inputs and don't want to continue if I am going to hit a limit at some point. Anyone know if there is an input limit? Thanks.

Comments

  • dthierjung
    dthierjung

    To my knowledge, there isn't a cap, however I've never gone beyond 5 inputs.

     

    I'm not familiar with your data landscape but taking a step back, it might be prudent to consider other ETL options given your estimation of inputs. That dataflow would be cumbersome at best to manage and make changes to, not to mention debug for errors and even fully execute.

     

    If possible, I'd  recommend trying to do some of the heavy lifting on-premise maybe in SQL environment.

  • guitarhero23
    guitarhero23 Contributor

    Can you provide some more info about your use case? I agree with the other post, it seems like you may be better off doing things somewhere else to bring into Domo


    **Make sure to like any users posts that helped you and accept the ones who solved your issue.**
  • AS
    AS Coach

    Agreed with responses above.  Even if you can I'd have to question if that's something you want to maintain. Maybe if you let us know about why you have 100 sources to begin with.  There may be solutions to pre-combine that data before loading into Domo.

    Aaron
    MajorDomo @ Merit Medical

    **Say "Thanks" by clicking the heart in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"

Categories