If you are using your encoder to measure the spindle speed, and the encoder looks like it is counting up or down nice and smooth, but the calculated RPMs are jumping all over the place, the likely issue is electrical noise. The quadrature encoder with its A and B channels will be able to ignore the noise and count up or down correctly (although you may see it bounce repeatedly between [just picking a number out of the blue] 262 and 263 before it starts bouncing between 263 and 264). However, the ESS is only looking at the A channel for it's RPM calculations, and every noise bounce on that signal input will count as an encoder pulse and cause a new RPM calculation with a period that was too short. In this situation, some noise filtering of 5 us, 10 us or 20 us will likely fix the issue so you can see stable RPM measurement - but you should still try and fix the root cause of the noise issue, not just ignore it withsoftware.
What is the BEST PPR (pulses per revolution) choice for an encoder input on a spindle? It is very, Very, VERY dependent on the RPMs you are running at and the number of pulses per revolution that you are running at. It also depends on if you are just milling with the spindle (a lower PPR will be fine) or if you are threading with it on a lathe where you will want more pulses per rev - especially at slower RPMs).
This chart tells you a lot, and that is that the middle ground is usually your best choice. However, usage specific requirements may pull you one way or the other from center. Trial and error will dictate what you need to do. When you see reported RPMs that are nice and stable (not much flutter) , which also respond quickly to changes in real RPMs you will have it dialed in where you want it.
If you wanted a 1000 count encoder, but could only get a 5000 count, you could just increase the prescalar factor some more to give fewer "effective pulses".
You can always prescale down a higher count encoder. you can't increase a 4 count per rev encoder.
A 100 count should work, but you may run it without any prescaling or just a divide by 2.
A 1000 count will work, but you will likely prescale by 2, 4, 5, 10 or 20. the 10 would give you 100 counts effectively, the 20 would give you 50 counts effectively, just like with the 100 count example. The 1000 ppr encoder gives you a lot more flexibility.
Just keep in mind, a million ppr encoder would overwhelm the ESS above 4 RPS or 240 RPM. There is an upper limit of 4 Million pulses per second (4 MHz) for the ESS inputs. Some BOBs also have input filtering that will reduce that bandwidth some more, but that is BOB specific.
I would think that the ideal scenario is 40 effective pulses to 100 effective pulses (<--- ideally) up to 250 effective pulses per second for the ESS to adjust to changing rotational velocities while threading. Going above 1,000 effective pulses would be detrimental, and I don't see any benefit to even going above 500 effective pulses per second.
Take your target RPM, multiply by the effective PPR of your encoder and then divide by 60 (sec/min) to get the resulting pulses per second (Remember that prescaling reduces the effective PPR of your encoder.)
So (1000 RPM * 10 PPR ) / ( 60 sec/min) = 166 pulses per second. Great!
Now if you do that for 500 RPM, you get 83 PPS I guess that is okay.
Do it at 100 RPM and you get only 16.6 PPS which is lower than the 40 effective pulses per second. However, that would still be 10 times better than just an index pulse.
The ESS -> Input signals tab tells the FPGA which input signals to enable (use) and which pins they are connected to. In your case, you are connecting (mapping) the Spindle Index input signal up to the pin you named as "INDEX". Any input signal that you want to use needs to be enabled and mapped to a pin.
Over on the ESS Config -> Spindle tab, we have the ability to select the source of an RPM signal, either the Spindle Index signal or the Spindle Encoder A signal.. 98% of all users (who read a RPM speed - my guess-) will just set that to their spindle index signal, since that is included with their hardware, and for mill and router users, this is the best choice!.
However about 2% of users will also have an encoder on their spindle because it is a A (or 4th or 5th) axis setup or a lathe. In the A axis setup, the user wants to be able to see their angular position which is why they have the encoder. In the lathe setup they may want to be able to see their angular position as well, but they may also turn large diameter items at slow speeds which means that they may receive less than 1 RPM per second (or commonly less than 10 RPM per second). An index pulse this infrequently is too granular and will not report a stable RPM. Worse yet is that when threading the ESS will compensate the Z and X speeds to match the instantaneous RPM. If an encoder is available, with say 1024 PPR, we could then set the RPM source to the encoder and calculate the instantaneous RPM speed up to 1024 times more often which results in much more consistent threads.
However, someone who is just running a router bit at 26,000 RPM will see zero improvement setting their RPM source to an encoder instead of just using the index signal.