Back
[01:15:41] <jepler> sigh, there's a bug in this "ring buffer" thing I wrote, and I don't understand it.
[01:16:25] <jepler> it has to do with reliably detecting that there's nothing pending, now that it can be "pending" in both the old kernel-side motion queue, and the userspace->kernel ring buffer
[01:16:32] <jepler> ... I think
[01:17:01] <cradek> sounds hairy
[01:18:27] <jepler> in HEAD, does the "manual toochange" window pop up only once the tool has reached the change location?
[01:18:44] <cradek> yes I'm pretty sure
[01:21:17] <jmkasunich_> hi guys
[01:21:24] <cradek> hi john
[01:21:32] <jepler> hi jmkasunich_
[01:21:35] <jepler> you have a _
[01:21:40] <jmkasunich_> hmm
[01:21:44] <jmkasunich_> jmkasunich_ is now known as jmkasunich
[01:21:50] <jepler> cradek: humor me and check?
[01:22:06] <jmkasunich> about that ring buffer...
[01:22:08] <cradek> arg, I'm crippled right now, installing dapper on a new disk
[01:22:40] <jepler> jmkasunich: I'm very interested in what you have to say about it
[01:22:45] <jmkasunich> I've wanted to move the main buffer to the user/kernel border for a long time
[01:23:16] <jmkasunich> the dual buffer approach is easier to implement
[01:24:14] <jmkasunich> the problem with moving the main buffer to the user/kernel interface is that some commands should be buffered and others should not
[01:24:56] <jepler> what do you mean, "the main buffer"?
[01:25:04] <jmkasunich> the TP motion queue
[01:25:43] <jmkasunich> I haven't actually looked at your code yet
[01:25:50] <jepler> nobody has but me
[01:25:59] <jmkasunich> not committed?
[01:26:02] <jepler> no
[01:26:06] <jepler> I'm not confident enough in it
[01:26:17] <jepler> as I was discussing earlier
[01:26:25] <jmkasunich> then how did I know about it? IRC maybe?
[01:26:31] <jepler> I had mentioned it on IRC, yes
[01:26:35] <jmkasunich> ok
[01:26:44] <jepler> last night I said something about it
[01:26:50] <jmkasunich> I think that was the "are you there or sleeping" thing I wanted to talk about last night
[01:27:39] <jmkasunich> so, right now, command.c gets called once per servo period, and can process one command per period ("process" means stick in the TP queue, for things like line and arc commands)
[01:27:45] <jmkasunich> did you change that?
[01:28:20] <jepler> let me see if I can summarize what I did
[01:29:11] <jepler> command.c still gets called once, but it will process items from this 'ring buffer' until there are no items left
[01:29:17] <jmkasunich> ok
[01:29:18] <jepler> (a bounded amount of work, because the ring buffer has a fixed size)
[01:29:37] <jmkasunich> right, but still can be an order of magnitude longer than before
[01:29:45] <jepler> yes, and I haven't done any timings
[01:30:17] <jepler> in 'task', you can 'put' to the ring buffer, which will block if full. I added some code in a few places so that it would not sleep if it was adding motions to the ring buffer.
[01:30:44] <jmkasunich> ok
[01:31:01] <jmkasunich> that part I figured out from the earlier conversation, and is great
[01:31:11] <jmkasunich> the part in kernel space is a compromize
[01:31:23] <jmkasunich> if you process everything in the buffer, the worst case execution time goes up
[01:32:00] <jmkasunich> if you process one entry per period, execution time remains small but commands like "abort" can get delayed
[01:32:42] <jepler> right, so choosing the ring buffer size, and perhaps limiting the number of items "dequeued" at a time, would be important. If the buffer is 32 items, and you dequeue up to 4 per ms, that's 8ms an abort could be delayed in the ring buffer.
[01:33:46] <jmkasunich> back when PeteV and I (and some others) were discussing things, we thought about having two buffers
[01:34:05] <jmkasunich> one would be the main TP queue, except things would be enqueued in user space, not kernel space
[01:34:28] <jmkasunich> the other would be a 1-deep buffer, like now, for commands that need executed immediately
[01:35:05] <jepler> the question I would ask is whether an 8ms delay is "important", when it's userspace that is adding items in the first place
[01:35:21] <jmkasunich> the answer might be no
[01:36:02] <jmkasunich> my approach _needed_ the "immediate command" buffer, because the main TP queue can have minutes worth of commands in it
[01:36:15] <jepler> ah
[01:36:22] <jepler> *lightbulb*
[01:36:33] <jmkasunich> your approach solves that by emptying the buffer every time (I think)
[01:36:50] <jmkasunich> what happens if the TP queue is full? you can't empty your ring buffer can you?
[01:37:14] <jepler> hm, I'm not sure.
[01:37:23] <cradek> userspace could watch, and never fill it up
[01:37:30] <jmkasunich> eww
[01:37:40] <jepler> brb (I hope)
[01:38:21] <jmkasunich> right now, the userspace interface is clean - if the ring buffer is full, block, else enqueue the command and keep going
[01:38:55] <cradek> yeah, my idea sucks
[01:38:56] <jmkasunich> if you have to say "will all the commands in the ring buffer fit in the main buffer" before deciding to enqueue a new one, that is messy
[01:39:29] <cradek> it's not that messy, you just subtract
[01:39:45] <jmkasunich> but that means you need to have hooks into the main TP buffer
[01:39:58] <jmkasunich> if you're going to go that far, why not just enqueue directly into that buffer
[01:40:01] <cradek> motion already has tcqLen() or somesuch
[01:40:04] <jmkasunich> in user space:
[01:40:18] <jmkasunich> if ( line/arc ) {
[01:40:27] <jmkasunich> if ( room in bufffer ) {
[01:40:35] <jmkasunich> enqueue()
[01:40:39] <jmkasunich> } else {
[01:40:41] <jmkasunich> block
[01:40:44] <jmkasunich> }
[01:40:53] <jmkasunich> } else {
[01:41:05] <jmkasunich> if ( room in immediate buffer ) {
[01:41:12] <jmkasunich> stick in buffer
[01:41:15] <jmkasunich> } else {
[01:41:17] <jmkasunich> block
[01:41:19] <jmkasunich> }
[01:41:20] <jmkasunich> }
[01:41:30] <jmkasunich> fscked up the indenting, but you get my drift
[01:41:34] <cradek> sure
[01:42:02] <cradek> brb after reboot
[01:46:07] <jepler> I'll have to continue this conversation later
[01:46:12] <jmkasunich> dang
[01:46:17] <jepler> sorry, life is calling me
[01:46:32] <jmkasunich> I need to get one of those one of these days
[01:46:39] <jepler> not this one
[01:46:46] <jmkasunich> oh...
[01:46:49] <jmkasunich> good luck then
[01:47:18] <cradek> yay
[01:47:29] <cradek> something wrong jepler?
[01:47:32] <jmkasunich> booted?
[01:47:39] <cradek> yeah the dapper install works great
[01:47:59] <cradek> it's a little tight on 256 meg, might not have made it if it hadn't found a swap partition to use
[01:53:15] <jepler> oh nothing that wouldn't sound petty if I bitchd about it on irc
[01:53:32] <cradek> ok, I understand
[01:53:50] <jmkasunich> are you back, or just passing by?
[01:54:14] <jepler> I think I'm back
[01:57:24] <jepler> I'm not sure what the current code does when the queue is full
[01:57:38] <jepler> I mean, before my changes
[01:58:01] <jmkasunich> hmm
[01:58:07] <jmkasunich> I thought I knew, but now I wonder
[01:58:23] <jmkasunich> I was going to say "it simply doesn't process the next one"
[01:58:34] <jmkasunich> implying that it doesn't handshake it, and userspace waits
[01:58:43] <jmkasunich> but what if userspace needs to send an abort
[01:58:47] <jmkasunich> * jmkasunich looks at code
[02:00:04] <jepler> it looks like it errors if the queue is full when it tries to add
[02:00:12] <jmkasunich> it does?
[02:00:36] <jepler> tcqPut <- tpAddLine <- emcmotCommandHandler
[02:01:24] <jmkasunich> I see
[02:01:41] <jmkasunich> that means user code must be checking for a full queue before it issues the command
[02:01:50] <jmkasunich> lame
[02:02:09] <jmkasunich> well, maybe lame
[02:02:16] <jepler> I think so
[02:02:28] <jepler> case EMC_TASK_EXEC_DONE:
[02:02:28] <jepler> STEPPING_CHECK();
[02:02:28] <jepler> if (!emcStatus->motion.traj.queueFull &&
[02:02:28] <jepler> emcStatus->task.interpState != EMC_TASK_INTERP_PAUSED) {
[02:02:28] <jepler> if (0 == emcTaskCommand) {
[02:02:30] <jepler> // need a new command
[02:02:33] <jepler> emcTaskCommand = interp_list.get();
[02:03:19] <jmkasunich> seems to me if the queue is "inside" the motion controller (rather than at its interface with user code), then it really should be _inside_
[02:04:10] <jmkasunich> but I guess that is neccessary, so that you don't have a command that needs to go in the queue but can't blocking an immediate command
[02:04:52] <jepler> (hm, 'queueFull' will be another thing you can't depend on, since it won't know about items in the ring buffer .. 'full' would instead mean 'less than ring_buffer items available in the queue or something stupid like that)
[02:05:02] <cradek> jepler: is there code in there for probe that works like pause?
[02:05:21] <cradek> (a tangent, I know)
[02:05:46] <jepler> case EMC_TRAJ_PROBE_TYPE: // prevent blending of this
[02:05:46] <jepler> case EMC_TRAJ_CLEAR_PROBE_TRIPPED_FLAG_TYPE: // and this
[02:05:46] <jepler> return EMC_TASK_EXEC_WAITING_FOR_MOTION_AND_IO;
[02:06:00] <cradek> cool
[02:06:04] <cradek> maybe it's not as broken as it seems
[02:08:21] <jmkasunich> jepler: I think we should investigate the queue + immediate buffer approach instead of adding the ring buffer
[02:08:30] <jepler> I can accept that
[02:08:39] <jepler> this conversation has shown me there are more problems than I anticipated
[02:09:33] <jmkasunich> the main problem with putting stuff directly in the TP queue will be making sure the enqueue/dequeue code is thread safe
[02:09:40] <jmkasunich> the existing code doesn't need to be
[02:10:09] <jmkasunich> command handler puts stuff in, then later in the same thread control (when needed) pulls it out
[02:10:43] <jepler> yeah
[02:10:50] <jmkasunich> I'm sure you had to do something like that for your ring buffer, it has the same issue
[02:11:41] <jmkasunich> heh:
[02:11:42] <jmkasunich> * \subsection TC queue functions
[02:11:42] <jmkasunich> * These following functions implement the motion queue that
[02:11:42] <jmkasunich> * is fed by tpAddLine/tpAddCircle and consumed by tpRunCycle.
[02:11:42] <jmkasunich> * They have been fully working for a long time and a wise programmer
[02:11:42] <jmkasunich> * won't mess with them.
[02:12:27] <jepler> perhaps I didn't think through the "thread safety" enough, and it works by good luck and the fact that it's not a full moon (is it?)
[02:12:37] <jepler> well, for posterity's sake:
http://emergent.unpy.net/index.cgi-files/sandbox/ringbuffer.patch
[02:12:57] <jmkasunich> the window of vulnerability is probably a few nS wide
[02:13:08] <jmkasunich> so it could work for days, weeks, months
[02:13:29] <jepler> +typedef struct {
[02:13:29] <jepler> + volatile int head, tail;
[02:13:29] <jepler> + volatile emcmot_command_t elem[SZ];
[02:13:29] <jepler> +} ring_buffer_t;
[02:13:55] <jepler> userspace only moves 'head' after copying data, and kernel space only moves 'tail' after it's done with the data
[02:14:18] <cradek> ha, I wrote that comment
[02:14:56] <jmkasunich> head is only every changed by userspace, and tail is only ever changed by kernel space, right?
[02:15:02] <jepler> right
[02:15:17] <jmkasunich> but both user space and kernel space need to compare head and tail to detect full/empth
[02:16:23] <jepler> it's OK if the kernel misses going from empty to non-empty, so it doesn't matter that it might miss seeing 'head' move
[02:16:40] <jepler> and it's OK if the user misses going from full to non-full, so it doesn't matter that it might miss seeing 'tail' move
[02:16:56] <jepler> but since we're junking this code it doesn't matter whether I got the details right
[02:17:10] <jepler> I just waste our time by explaining how confident I was when I wrote it
[02:17:43] <jmkasunich> I think you had reason to be confident (at least as far as the ring buffer itself)
[02:18:11] <jmkasunich> the interaction of the two buffers is where it gets funny
[02:18:40] <jmkasunich> and I have reservations about the variable execution time with more than one entry in the buffer
[02:18:43] <jepler> and that's what I was starting to discover with this toolchange business
[02:19:40] <jmkasunich> pre-conditions and post-conditions really complicate things
[02:20:22] <jmkasunich> I remember talking to Fred at NIST and he said if he was doing it over, he'd probably do pre and post conditions differnetly
[02:20:38] <jmkasunich> basically, queue up additional commands, instead of attaching conditions to commands
[02:20:53] <jmkasunich> commands like "wait for motion complete"
[02:21:41] <jmkasunich> then just have the canonical code issue the appropriate combination
[02:22:07] <jepler> this goes beyond my vision
[02:22:40] <jmkasunich> yeah, definitely
[02:22:51] <jmkasunich> I never think small ;-/
[02:25:12] <jepler> I'm going to call it a night
[02:25:33] <jepler> jmkasunich: thanks for taking the time
[02:29:41] <jmkasunich_> dang... I need a UPS
[02:31:17] <jmkasunich_> did I miss anything?
[02:31:32] <cradek> 21:33:29 < jmkasunic> I never think small ;-/
[02:31:32] <cradek> 21:35:50 < jepler> I'm going to call it a night
[02:31:32] <cradek> 21:36:10 < jepler> jmkasunich: thanks for taking the time
[02:31:52] <jmkasunich_> thanks
[02:32:07] <jmkasunich_> the power failure was only 2 seconds or so... but enough to reboot everything
[02:32:20] <cradek> is that common in your area?
[02:32:26] <jmkasunich_> only during thunderstorms
[02:32:40] <cradek> ah
[02:32:55] <cradek> we get probably a couple a year
[02:33:09] <jmkasunich_> we had one a few days ago, and this one
[02:33:11] <cradek> enough to warrant a UPS but not by much
[02:33:23] <jmkasunich_> I think thats all I've had so far this summer
[02:33:50] <cradek> they're a pain
[02:35:25] <cradek> dapper looks nice
[02:35:35] <cradek> (I guess I shouldn't be surprised)
[02:39:42] <jmkasunich_> jmkasunich_ is now known as jmkasunich
[13:42:31] <rayh> dr dobbs this morning is telling me about a a webcast (which I can't get of course) about agile software.
[13:43:03] <rayh> the post regarding it says
[13:43:11] <rayh> "will expose a surprising opportunity to add more agility to the development process – software builds. The oft-overlooked (and constantly growing) build process is vital to the ability to obtain rapid feedback throughout the development process - a key contributor to agility. The presentation will discuss how tools and processes applied to software builds can improve a team’s ability to react to change, improve software quality, and achieve fa
[13:43:12] <rayh> ster time to market.
[13:45:06] <jepler> I scoffed recently when I read on "freshmeat.net" about a proprietary product that seemed to have as its purpose "build your software once in awhile, just to make sure it actually compiles"
[13:46:51] <rayh> As many times as I've had compile failures with emc it seems on the surface that compiling in the wild would be a bad idea.
[13:47:26] <jepler> I agree, but it doesn't seem like "rocket science" to do it
[13:47:57] <jepler> it's not a development paradigm and it hardly warrants shelling out for proprietary software
[13:48:00] <jepler> bye ray
[13:54:08] <rayh> That "outage" was much to close to the hour to be coincidence!
[13:54:29] <rayh> you were saying, jepler?
[13:54:50] <jepler> 08:58:07 <jepler> I agree, but it doesn't seem like "rocket science" to do it
[13:54:54] <jepler> 08:58:38 <jepler> it's not a development paradigm and it hardly warrants shelling out for proprietary software
[13:56:47] <rayh> Absolutely. You do it every day.
[14:02:36] <rayh> The thought that struck me was the idea that instead of reading a bunch of ini variables and setting up HAL, what would happen if we had an "agile" enough build system to produce a custom set of executables.
[14:03:08] <rayh> A bit like the kernel build system.
[14:06:32] <jepler> on the other hand, I'm pleased we can offer more functionality without recompiling
[14:06:40] <jepler> that's why I made the kinematics modular, for instance
[14:07:23] <jepler> most users aren't interested in recompiling software
[14:08:15] <rayh> Sure. EMC moves toward a scripting thing rather than a compiling thing.
[14:09:36] <rayh> One think that Paul and I were thinking about, before he took a hike, was the idea of minimal sized embedded systems.
[14:10:05] <rayh> There it seemed like a compile would produce a smaller footprint that a connect and go setup.
[14:10:15] <rayh> But that may not be true.
[14:12:13] <jepler> yeah, the rules are different embedded, dedicated systems
[14:12:28] <rayh> You'd have a much better feel for the space required to run an EMC than I.
[14:13:12] <rayh> and as some point out even in embedded systems the size and power rules have relaxed a lot.
[14:13:59] <jepler> du --total rtlib bin lib -> about 5.5 megs excluding lib/python
[14:15:50] <rayh> back in the day, I got a RedHat 5.1 version of the EMC compile onto a floppy.
[14:16:01] <jepler> nice
[14:16:13] <rayh> Not a lot left.
[14:17:18] <rayh> if we aimed at something like only Chris's lathe or only a stepper 3 axis mill, we could cut quite a few files out still.
[14:21:52] <jepler> here's where "embedded" seems to be these days: x86, 32-256MB SDRAM, 8MB-4GB compactflash "disk" (not included), $213 qty 1:
http://www.soekris.com/net4801.htm
[14:22:44] <rayh> looking.
[14:23:25] <jepler> you can get ubuntu "server" install + emc2 in a 512MB compact flash easy .. it's almost not worth worrying about the size, it seems to me
[14:24:08] <jepler> (I'm not saying this particular system is suitable for emc, I don't know one way or the other)
[14:24:29] <rayh> Okay. Thanks for the link and the assessment.
[14:24:56] <jepler> I'm full of opinions :-P
[14:24:57] <jepler> bbl
[14:25:17] <rayh> I did get coolcnc on a 128M usb stick
[14:46:40] <jepler> neat
[16:49:46] <SWPadnos> hi folks
[16:50:18] <SWPadnos> rayh, one thing kind of between JIT compiling and scripting would be closer to JIT linking (which is what HAL does for us, sort of)
[16:59:20] <rayh> Hi Steven. Okay I kinda understand.
[17:01:57] <SWPadnos> heh
[17:02:34] <SWPadnos> in a sense, the last stage of compiling is like HAL - the linker takes the separate object files and puts them together into a single executable
[20:35:53] <lilo> [Global Notice] Hi all. If you've been experiencing problems with resetting nicknames, please message me. We've located the issue and should be able to resolve it for you pretty quickly. Thanks.
[20:37:36] <lilo> [Global Notice] Sorry, let me rephrase that....if you've had problems with your cloak 'spontaneously' changing, please message me. Thanks.