Production/workflow systems

Discussion in 'Game Development (Technical)' started by voxel, Dec 22, 2006.

  1. voxel

    voxel New Member

    Joined:
    Oct 29, 2006
    Messages:
    176
    Likes Received:
    0
    As I start gather the pieces of my new studio, I realized I need to build a game-oriented production pipeline to facilitate collaboration with external contractors. It'll probably be web-based and just functional (i.e not pretty). Anybody build one before? Or is there one being developed publicly?

    I'm thinking of:

    1) Drupal + WebDAV + a whole bunch of custom code to allow artists to post designs, textures, finished 3D scenes, set approval levels, etc.

    OR

    2) A custom Ruby on Rails system. The only benefit of this is a well-defined SQL schema that can be used by tools to process game data. (1) is easier to start off with and provides a nicer workflow for artists.

    The system should have "approval levels", "asseting groups" (levels, sets, etc.), a method to access the data via SQL or an API. I don't want a CVS/Alienbrain/Subversion asset repository - been there(all three). done that. sucks ass. Linking to files is fine by me.

    I've worked with media workflow systems in OTHER industries, but not in the game biz. So far, only Alienbrain comes close, but it's too server-side centric and is pretty expensive. Anybody want to collaborate on a CMS for games?
     
  2. Paul-Jan

    Original Member

    Joined:
    May 30, 2006
    Messages:
    60
    Likes Received:
    0
    Slightly offtopic, but if you could find the time to elaborate a bit on the main problem areas with a subversion repository for this purpose, I'd most appreciate it. I'm just curious. Basically that would also be a nice highlight of the must-haves for your game-oriented CMS.
     
  3. voxel

    voxel New Member

    Joined:
    Oct 29, 2006
    Messages:
    176
    Likes Received:
    0
    The problem with Subversion + CVS + Alienbrain is their insistence in obfuscating the underlying files. It's like how iTunes mangles the MP3 filenames when you import them. What I want is a decoupling of the metadata (i.e versioning) from the storage mechanism so I can

    1) Backup the files using a system (i.e rsync) separate from SVN/CVS/Alienbrain.

    2) Access the files through HTTP/WebDAV/NFS/Mel/MaxScript/Curl/SOAP etc... which are more artist-friendly than CVS/SVN UIs.

    In 3D terms, it's like referencing in Maya and XREFing in 3DS Max. Linking to external sources is way more flexible + robust than importing.

    And versioning files is the least critical part of the workflow system. Storing the connections between art assets is the primarily goal. What are all the models in this level? What are all the swords the elf can hold? What materials/textures are referenced by this prop? Only Alienbrain (when I last used it) provided these types of meta-query functionality.

    We have all these XML files pumped out by the art exporters, but no linkage between them. Bunch of models here. Bunch of textures there. An assortment of animations. The "connections" are stored in the level files (i.e Maya scenes) and modified by very complicated level editors.

    The goal is to have a centralized repository for game metadata that are used by:

    1) Web browsers + servers (for approval and asset tracking)
    2) Art tools (to determine what needs to be exported and processed)
    3) Game engine (to determine what needs to be loaded, etc. - probably needs a preprocessing step).

    I know the system sounds like overkill, but I'm on a project that has only 3 artists and I'm having trouble getting all art sync-ed (data overload). Maybe I've been pampered in the past. I've been on 100+ artist productions (multiple remote studios) that have worked SMOOTHLY with custom workflow solutions. Unfortunately, I have minimal web-programming experience and will probably spend less than a month prototyping something then open sourcing it.
     
  4. Nutter

    Original Member

    Joined:
    Sep 15, 2004
    Messages:
    18
    Likes Received:
    0
    wow... You're underestimating the importance of version control of assets by a huge amount!

    The best solution is unfortunately custom right now (as it appears you've already figured out), however it should definitely be integrated with a good version control system. If you have the budget, I recommend using Perforce as it not only is the #1 version control software (use it for both source and assets), but also has a library that you can use to make plugins for Max, Maya, Photoshop, and all your custom tools.

    Having said that though, the kind of system that you're talking about is overkill to the extreme if you only have 3 artists - the amount of time wasted by artists "doing it manually" will pale in comparison to the amount of programmer time spent on a complex all-in-one solution for all the tools that your team will be using.
    Pick your fights carefully - if you do all these "nice haves" before starting on your actual game, you may well find that you're eating ramen noodles 6 months before you'll be seeing any income from the game. Get something out there so you have at least some cash flow, then re-evaluate the asset-pipeline situation as the content creators may well have no problem working with the klunky tools after a bit of learning.
     
  5. voxel

    voxel New Member

    Joined:
    Oct 29, 2006
    Messages:
    176
    Likes Received:
    0
    No am not. Been there. Done it. It's NOT what makes a workflow system great. I've worked at game studios where you typed a command and all data gets processed, all code gets compiled, and a final game gets built in 10-20 so hours. I've worked at animation studios where you click a button - entire shots get assembled, animation + constraints + lighting + lipsync get applied and sent off to the render farm. I've even worked at a media asset software company once. Total BS. Version control of assets is like number 10 on my list. Centralizing how data is inter-connected (aka metadata) for all the tools and describing how + what should be built (i.e using "makefiles" for art) are my primary goasl.

    There are visual "makefile" systems for art - but they are very, very expensive - $200+ k.

    Yes for 3 artists a workflow system is a bit overkill - I'm using a modified version of Drupal at the moment for a contract job. My Maya tools don't communicate with it so I'm left with a random collection of XML and CSV files that magically combine in the game engine - that no artists understands nor can edit.

    I don't plan to spend more than a month on dev. - so I am picking my battles. I've kinda started also - a Ruby on Rails system (started learning it). I plan to have Maya tools/exporters directly connect to the database via some REST/SOAP/XML-RPC system. Web programming is not my strongest point, but I'll build something. The game engine will use the same metadata - might need to be pre-compiled into a binary form first tho.

    The biggest reason I NEED to do this is to help me work with off-site artists in the future. I estimate I'll have 6-10 artists - mostly part-time. I really don't want to hire anybody full-time to be honest. I prefer to contract all the work out - and make it so easy that I don't have to "train" people all the time.

    I've also limited the toolset to Maya, Firefox, Photoshop/ImageMagick, Perl/Python, Audacity, OGRE, and CEGUI. And I'll bang out two or three "previewer" apps that the artists will use to test their work before submitting it to me via the workflow system. What I want is well-understood framework and pipeline before I go into material development.

    I'm not worried about cashflow - all dough I've made contracting my soul out can last me a few years.
     
  6. zoombapup

    Moderator Original Member

    Joined:
    Nov 25, 2004
    Messages:
    2,890
    Likes Received:
    0
    You do understand that subversion works with HTTP right? It can use webdav and whatnot as well if I recall.

    I think youre talking more about a build system than a workflow system. I'd be tempted to throw together a subversion client that does all of the stuff you want to do and throws links into a central directory somewhere (so for your artist, it would sanity check stuff, then throw it into subversion and spit out a link of whatever flavour into a directory so that other systems can see it).

    I really wouldnt start pissing about with too complex a thing though.. ugh.
     
  7. voxel

    voxel New Member

    Joined:
    Oct 29, 2006
    Messages:
    176
    Likes Received:
    0
    I know Subversion uses WebDAV. WebDAV is great except certain language bindings are crap/buggy from personal experience.

    Subversion mangles the underlying files so it's hard to get distributed repositories to sync up correctly. Distributed systems are different than monolithic systems - similar to single database vs. multi-base. Also, Subversion is also quite slow for large file.

    For now Drupal and FTP are fine for artists...

    It's not a "build system" at all. I want to:

    1) Track art approvals + designs (i.e like Drupal)
    2) Store and access art (like Subversion)
    3) Allow tools to access metadata(via HTTP or SQL) to build assets (i.e Geometry converter, DXT conversion, etc.)

    I guess it's hard to describe something only available in big studios or for lots of $$$. To me - tracking files is LESS important than tracking the status/comments/notes of all the art pieces. Funny hearing that from a programmer, eh? Fancy file versioning software is pretty un-important because in the end-goal is collaboration.

    This is a build system: http://www.temerity.us/home.php
    GroupWare systems, CMSes and project managment systems almost are usable for tracking art...

    Anyhow, I refuse to re-invent the wheel (refraining from writing an engine or art exporters). My goal is to have a virtual studio collaborating with other small studios and freelancers. Right now, many studios work via e-mail, FTP, CVS - which fine for small teams and limited amounts of data.

    I'm not spending any significant time on this yet - designing a game, budgeting, and setting up the company...

    I'm looking at this 3rd party system (if I get seed funding)

    http://sthpw.com/features/index.htm

    But it might need to be massaged for games.
     

Share This Page

  • About Indie Gamer

    When the original Dexterity Forums closed in 2004, Indie Gamer was born and a diverse community has grown out of a passion for creating great games. Here you will find over 10 years of in-depth discussion on game design, the business of game development, and marketing/sales. Indie Gamer also provides a friendly place to meet up with other Developers, Artists, Composers and Writers.
  • Buy us a beer!

    Indie Gamer is delicately held together by a single poor bastard who thankfully gets help from various community volunteers. If you frequent this site or have found value in something you've learned here, help keep the site running by donating a few dollars (for beer of course)!

    Sure, I'll Buy You a Beer